AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression
Yijing Chen, Wenhui Tan, Xiaoyi Yu, Yuyue Wang, Xin Cheng, Kaisi Guan, Hao Jiang Xiangyang Li Guojie Zhu Ruihua Song · 2026
Read AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression by Yijing Chen, Wenhui Tan, Xiaoyi Yu, Yuyue Wang, Xin Cheng, Kaisi Guan, Hao Jiang Xiangyang Li Guojie Zhu Ruihua Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24286.txt.
Description
Chen and colleagues help omni-modal language models watch hour-long audio-video content without drowning in tokens. Long recordings overflow context windows and repeat themselves endlessly, so most tokens carry nothing new. Their 2026 framework, AVOC, inserts a learnable compression module between the audio-video encoders and the language model, treating compression as a retrieval problem under a fixed budget.
Three classical information-retrieval criteria guide which tokens survive: relevance to the user's query, general importance, and diversity among the chosen set. Each criterion becomes a tailored mechanism inside one unified pipeline, so the retained subset best supports answering the actual question rather than summarising everything equally. On long-form audio-video benchmarks AVOC reaches state-of-the-art performance, beating the second-best model by several points of average accuracy on OmniVideoBench and a companion benchmark. Chen's retrieval framing turns an engineering squeeze into a principled selection problem.
Catalog record
| CREATOR | Yijing Chen, Wenhui Tan, Xiaoyi Yu, Yuyue Wang, Xin Cheng, Kaisi Guan, Hao Jiang Xiangyang Li Guojie Zhu Ruihua Song |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24286