RAPID: Retrieval-Augmented Parallel Inference Drafting for Text-Based Video Event Retrieval
Long Nguyen, Huy Nguyen, Bao Khuu, Huy Luu, Huy Le, Tuan Nguyen, Tho Quan · 2025
Read RAPID: Retrieval-Augmented Parallel Inference Drafting for Text-Based Video Event Retrieval by Long Nguyen, Huy Nguyen, Bao Khuu, Huy Luu, Huy Le, Tuan Nguyen, Tho Quan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16303.txt.
Introduction
Long Nguyen and co-authors present a system from two thousand twenty-five for finding events in video using text queries. Video collections grow fast, and users often describe what they seek in vague terms. Older search methods focus on objects in the frame and miss context, such as place or background, above all when the query is thin.
Their system, called RAPID, uses large language models to correct and enrich the query with missing context. Several enriched versions then search in parallel, and a final step picks the results that best match the original request. Tests on a custom dataset showed clear gains over standard methods for incomplete queries. The system also proved itself in a city AI contest over hundreds of hours of video.
Catalog record
| CREATOR | Long Nguyen, Huy Nguyen, Bao Khuu, Huy Luu, Huy Le, Tuan Nguyen, Tho Quan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-16303
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RAPID: Retrieval-Augmented Parallel Inference Drafting for Text-Based Video Event Retrieval Long Nguyen, Huy Nguyen, Bao Khuu, Huy Luu, Huy Le, Tuan Nguyen, Tho Quan Abstract Retrieving events from videos using text queries has become increasingly challenging due to the rapid growth of multimedia content. Existing methods for text-based video event retrieval often focus heavily on object-level descriptions, overlooking the crucial role of contextual information. This limitation is especially apparent when queries lack sufficient context, such as missing location details or ambiguous background elements. To address these challenges, we propose a novel system called RAPID (Retrieval-Augmented Parallel Inference Drafting), which leverages advancements in Large Language Models (LLMs) and prompt-based learning to semantically correct and enrich user queries with relevant contextual information. These enriched queries are then processed through parallel retrieval, followed by an evaluation step to select the most relevant results based on their alignment with the original query. Through extensive experiments on our custom-developed dataset, we demonstrate that RAPID significantly outperforms traditional retrieval methods, particularly for contextually incomplete queries. Our system was validated for both speed and accuracy through participation in the Ho Chi Minh City AI Challenge 2024, where it successfully retrieved events from over 300 hours of video. Further evaluation comparing RAPID with the baseline proposed by the competition organizers demonstrated its superior effectiveness, highlighting the strength and robustness of our approach. 1 1 footnotetext: The two authors contributed equally to this work. 1 Introduction The rapid advancement of Artificial Intelligence (AI), particularly in the field of Computer Vision, has driven significant breakthroughs in video event retrieval. Initially, research in this area focused on applications in Intelligent Transport Systems [ 1 ] , but it has since expanded to include surveillance, security, and multimedia content analysis [ 2 ] . With the exponential growth of video data across online platforms, the ability to efficiently retrieve specific events from vast video datasets has emerged as a critical research challenge [ 3 ] . Developing more efficient and accurate retrieval methods is essential to managing the overwhelming volume of media content generated daily and improving retrieval accuracy [ 4 ] . To address the growing demand for efficient video retrieval solutions, several international competitions, such as the Lifelog Search Challenge (LSC) [ 5 ] and the Video Browser Showdown (VBS) [ 6 ] , have been organized to foster the development of fast and effective video search methods. Similarly, in Vietnam, the Ho Chi Minh City AI Challenge 2024 1 1 1 http://aichallenge.hochiminhcity.gov.vn/ was established to encourage innovation in video event retrieval. In this competition, participants were tasked with retrieving events from over 300 hours of 1,500 news videos using a series of sequential text-based queries or short visual clips depicting the events. Notably, these queries often began with broad, context-poor descriptions, and many video segments lacked clear contextual cues, such as background details, making accurate retrieval a significant challenge. Various approaches to video event retrieval have been explored over the years, leveraging a diverse range of input modalities. These include image frames [ 7 ] , object tags [ 8 ] , event labels [ 9 ] , and even audio data [ 10 ] . More advanced methods use high-level sketches to visually represent events [ 11 ] . Recent progress in Natural Language Processing, especially with the development of multimodal models like Contrastive Language-Image Pre-training (CLIP) [ 12 ] , has significantly advanced text-based video retrieval. These models map text queries and video frames into a shared embedding space using contrastive learning, enabling more intuitive and effective retrieval [ 13 , 14 ] . However, the effectiveness of text-based retrieval systems heavily depends on the completeness of the query. Queries that lack sufficient context often lead to reduced retrieval accuracy [ 15 ] , underscoring the need for enriched contextual information [ 16 ] . Moreover, certain events are complex and difficult to describe in detail, further complicating the retrieval process. To address these challenges, we propose Retrieval-Augmented Parallel Inference Drafting (RAPID), a novel method designed to enhance text-based video event retrieval, particularly in scenarios where queries lack sufficient context and where frames are difficult to describe. Unlike traditional methods that rely solely on single, unrefined queries, RAPID generates multiple augmented queries by enriching the original with additional contextual details, such as location or event-specific information. These drafts are created using Large Language Models (LLMs) combined with prompt-based learning techniques [ 17 ] . By performing parallel inference , RAPID retrieves the most relevant keyframes, which are subsequently re-evaluated for alignment with the original query. Additionally, we developed a user-friendly interface to support practical, real-world application of the system. We evaluated RAPID using a dataset derived from 300 hours of news videos provided by the challenge. Our experimental results demonstrate that enriching queries with contextual information, such as location, significantly improves retrieval accuracy and efficiency. Furthermore, RAPID outperformed the baseline retrieval pipeline provided by the competition organizers, showcasing its ability to handle real-world video retrieval challenges, especially when dealing with incomplete or context-poor queries. In summary, our key contributions are as follows. • We developed the RAPID system, which leverages parallel processing of multiple augmented queries to significantly improve video event retrieval performance. • We demonstrated the effectiveness of RAPID in handling incomplete or context-poor queries, and compared its performance against the baseline provided by the competition, showing notable improvements. • We successfully applied RAPID in a real-world scenario, achieving excellent results at the Ho Chi Minh City AI Challenge 2024, highlighting its accuracy and speed. 2 Related Work 2.1 Text-based Video Event Retrieval Methods Various video event retrieval methods have evolved from traditional text-based queries that match against metadata such as titles and tags [ 8 , 18 , 9 ] . These methods often fail when metadata is incomplete or when users need to locate complex events that are difficult to describe using simple tags or titles. To address these limitations, visual-based retrieval techniques, such as content-based image retrieval [ 7 ] and query-by-high-level sketches [ 19 , 11 ] , were introduced to better analyze the visual content of videos. With the notable success of multi-modal models like CLIP [ 12 ] , baseline systems that retrieve both text and image frames from videos in a shared vector space have become increasingly effective [ 14 , 20 ] . These advancements have enabled more intuitive and accurate retrieval processes, especially when handling large-scale datasets. However, despite the improved performance of modern retrieval systems, challenges remain, particularly in aligning different modalities. This process requires substantial computational resources and poses scalability challenges for real-time applications [ 21 ] . Moreover, these methods heavily rely on the accuracy and completeness of text queries as the primary alignment mechanism. When event descriptions are vague or imprecise, system effectiveness is reduced. Research has shown that including contextual components in queries, such as location information, can significantly enhance model performance, as highlighted in [ 15 , 16 ] . 2.2 Augmented Query Strategies for Video Event Retrieval The rapid evolution of LLMs, combined with the power of prompt-based learning across various tasks [ 17 ] , has driven significant interest in developing more sophisticated systems for information retrieval [ 22