CA 2 ST: Cross-Attention in Audio, Space, and Time for Holistic Video Recognition
Jongseo Lee 1 , Joohyun Chang 1 , Dongho Lee, Jinwoo Choi 2 · 2025
Read CA 2 ST: Cross-Attention in Audio, Space, and Time for Holistic Video Recognition by Jongseo Lee 1 , Joohyun Chang 1 , Dongho Lee, Jinwoo Choi 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23447.txt.
Description
Naming the action in a video takes both a sense of space and a sense of time, and most models lean too hard on one. Four authors build a transformer system with separate spatial and temporal experts that swap information at every layer through a bottleneck cross-attention step. Each side's judgment is thus informed by the other, while the exchange stays cheap.
A fuller version adds a third expert for sound, letting hearing inform seeing. Tested on standard action benchmarks, the vision-only system performed in a balanced way across datasets that stress different skills. The audio-visual version also did well on four benchmarks that include sound. The authors present the design as a route to video recognition that treats sight and sound as partners, and they show the cross-attention module is where the benefit is created.
Catalog record
| CREATOR | Jongseo Lee 1 , Joohyun Chang 1 , Dongho Lee, Jinwoo Choi 2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23447