FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries
Qijie You 1 Hao Liang 2,4 Mingrui Chen 3 Bohan Zeng 2 Meiyi Qiang 2 Zhenhao Wong 2 Wentao Zhang 2,4 1 University of Science and Technology Beijing 2 Peking University 3 Institute of Automation, Chinese Academy of Sciences 4 Zhongguancun Academy · 2026
Read FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries by Qijie You 1 Hao Liang 2,4 Mingrui Chen 3 Bohan Zeng 2 Meiyi Qiang 2 Zhenhao Wong 2 Wentao Zhang 2,4 1 University of Science and Technology Beijing 2 Peking University 3 Institute of Automation, Chinese Academy of Sciences 4 Zhongguancun Academy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10228.txt.
Introduction
Finding a moment inside a long video from a short, vague query is harder than matching a caption. Qijie You and six colleagues built a test for exactly that gap. Their 2026 paper introduces FLARE, a benchmark for retrieving long videos using both sight and sound, from user-style queries rather than polished descriptions.
FLARE draws on 399 screened videos running 10 to 60 minutes each, totaling 225.4 hours. It holds 87,697 clips with vision, audio, and combined captions, plus 274,933 user-style queries. Some queries are filtered so that neither sight nor sound alone can find the answer. Tests of 15 retrievers showed that realistic queries change model behavior sharply, and that audio-language alignment remains the key bottleneck.
Catalog record
| CREATOR | Qijie You 1 Hao Liang 2,4 Mingrui Chen 3 Bohan Zeng 2 Meiyi Qiang 2 Zhenhao Wong 2 Wentao Zhang 2,4 1 University of Science and Technology Beijing 2 Peking University 3 Institute of Automation, Chinese Academy of Sciences 4 Zhongguancun Academy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10228
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries Qijie You 1 Hao Liang 2,4 Mingrui Chen 3 Bohan Zeng 2 Meiyi Qiang 2 Zhenhao Wong 2 Wentao Zhang 2,4 1 University of Science and Technology Beijing 2 Peking University 3 Institute of Automation, Chinese Academy of Sciences 4 Zhongguancun Academy Abstract As video becomes increasingly central to information dissemination and multimodal large language models (MLLMs) continue to advance, evaluating video retrieval has become increasingly important. In realistic search scenarios, this requires matching short user queries to long-form content using both visual and auditory evidence. Yet existing retrieval benchmarks are still dominated by short clips, single modalities, and caption-based evaluation. We introduce FLARE, a full-modality long-video audiovisual retrieval benchmark with user-simulated queries. Built from 399 carefully screened Video-MME videos (10–60 min, 225.4 h) to ensure source quality and diversity, FLARE contains 87,697 clips annotated with vision, audio, and unified audiovisual captions, together with 274,933 user-style queries. Cross-modal queries are further filtered by a hard bimodal constraint, requiring retrieval to fail under either modality alone but succeed when both are combined. FLARE evaluates models under two regimes, caption-based and query-based retrieval, across vision, audio, and unified audiovisual settings. Experiments with 15 representative retrievers show that user-style queries substantially change model behavior, strong caption-based performance does not always transfer to query-based retrieval, and audio–language alignment remains a key bottleneck for unified audiovisual retrieval. Our code and data are released at https://flarebench.github.io/ FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries 1 Introduction Video has become one of the dominant media for information dissemination on the modern Internet, and building high-quality video retrieval systems is therefore an increasingly important task. The rapid progress of multimodal large language models [ 26 , 33 ] has further elevated this need: video retrieval is now considered a key capability of MLLMs, since it directly reflects how well these models align long visual and auditory streams with natural-language intent. In a realistic search scenario, queries are fundamentally audiovisual (depending on both modalities), long-context (the target lies within a video spanning tens of minutes), and user-shaped (much shorter and more partial than a full description). However, public benchmarks still fall short of these properties: they focus on short clips, evaluate one modality at a time, and measure models against detailed captions rather than realistic queries. Most existing video–text and audio–text benchmarks were built around independently collected short clips, with later work mainly scaling up data. LoVR [ 2 ] took a step toward long videos but remains single-modal, while audiovisual benchmarks instead target QA [ 11 , 3 ] , classification [ 5 ] , or composed retrieval [ 14 ] —none of which probes whether unified multimodal models can actually retrieve long videos from realistic queries. Equally important, all of these benchmarks evaluate models against the same form of text used during construction, leaving it unclear whether reported gains transfer to the shorter, partial queries that real users issue. To fill this gap, we introduce FLARE , the first F ull-modality L ong-video A udiovisual R etrieval b E nchmark with user-simulated queries. FLARE comprises 399 long-form videos (10–60 min, 225.4 h) from Video-MME [ 11 ] , segmented into 87,697 clips, each annotated with vision, audio, and unified audiovisual captions, plus 274,933 user-style queries across three modality types. Every cross-modal query satisfies a hard bimodal constraint (Eq. 6 ): it must fail unimodal retrieval and only succeed when both modalities are combined, ensuring genuine audiovisual reasoning. Built on top of these annotations, FLARE further introduces a dual-regime evaluation protocol that benchmarks every model under both a caption-based regime (detailed captions) and a query-based regime (user-style queries) on the same gallery, isolating the impact of query formulation on model assessment. Benchmarking 15 representative contrastive and LLM-based retrievers [ 22 , 27 , 7 , 29 , 16 , 10 , 31 , 20 , 8 , 32 , 12 , 35 , 28 , 24 ] reveals two phenomena that demonstrate the value of FLARE. First, switching from captions to user-style queries causes the previously top-performing model in each of the three modalities to suffer a drop of around 20–40% in Recall@1, showing that user queries, although derived from the same content, pose a much greater challenge than information-rich captions and thus constitute an indispensable complement to caption-based evaluation. Second, comparing single-modal and full-modality settings reveals that audio retrieval is consistently the bottleneck of unified audiovisual models; strengthening audio–language alignment and designing better cross-modal fusion mechanisms thus remain important open challenges that single-modality benchmarks fundamentally cannot expose. Our contributions are summarized as follows: • We present FLARE, which, to the best of our knowledge, is the first full-modality long-video retrieval benchmark. A highly automated generation pipeline, complemented by rigorous human-assisted review, yields a high-quality benchmark of 399 long-form videos and 87,697 clips with vision, audio, and unified captions that jointly supports vision, audio, and unified audiovisual retrieval. • We further construct 274,933 user-style queries across three modality types, making FLARE, to the best of our knowledge, the first benchmark to introduce user-simulated queries with a hard bimodal constraint. Each query undergoes a rigorous validation process, ensuring that single-modality queries remain strongly relevant to the original captions while reflecting user-like search behavior, and that cross-modal queries further require information from both modalities. • We comprehensively evaluate 15 baselines and reveal differences and deficiencies at two levels— caption vs. query and single-modal vs. unified audiovisual —highlighting the limitations of current research. 2 Related Work 2.1 Single-Modality Retrieval Benchmarks Earlier video–text and audio–text retrieval benchmarks, such as MSR-VTT [ 34 ] , MSVD [ 4 ] , and VATEX [ 30 ] on the visual side, and AudioCaps [ 15 ] , Clotho [ 9 ] , MusicCaps [ 1 ] , and WavCaps [ 19 ] on the audio side, were largely built on independent short clips, with subsequent improvements mainly focusing on data scale and diversity. Such settings lack long-video context as a source of difficulty and are not well aligned with realistic search scenarios. With the rapid development of multimodal LLMs, recent work represented by LoVR [ 2 ] extends visual retrieval to long videos and introduces a more challenging temporal context, but it still operates within a single modality. As unified multimodal models become a central trend, building a video–text retrieval benchmark that simultaneously supports single-modality and unified-modality evaluation has become increasingly important. 2.2 Joint Audiovisual Benchmarks As unified multimodal models continue to advance, evaluating a model’s joint audiovisual capability has become a central focus of multimodal research. Some existing cross-modal benchmarks, such as Video-MME [ 11 ] and JointAVBench [ 3 ] , concentrate on QA-style evaluation of joint audio–visual reasoning, evaluating MLLMs via multiple-choice questions rather than embedding-based retrieval. Others, such as VGGSound [ 5 ] and OmniCVR [ 14 ] , focus on audio classification, where vision is treated more as auxiliary information, or on composed retrieval tasks where the model retrieves a target video given a source video and a textual modification. In contrast, we construct the first long-video retrieval benchmark that jointly covers vision, audio, and unified audiovisual settings, and we further build a user-style query dimension on top of detailed