X-Aligner : Composed Visual Retrieval without the Bells and Whistles
Yuqian Zheng, Mariana-Iuliana Georgescu · 2026
Read X-Aligner : Composed Visual Retrieval without the Bells and Whistles by Yuqian Zheng, Mariana-Iuliana Georgescu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16582.txt.
Introduction
Yuqian Zheng and Mariana Georgescu build a composed video search tool in plain form. Their 2026 preprint, named X-Aligner, deals with search by pic plus words. A user gives a clip cue and a word cue as one ask. Past tools fuse the two cues in one step at once. That one step fusion gave just small gains past base tools.
A staged align of cues aims for more large gains. Each stage lines cues up in a more fine way. Plain form with strong scores would ease real use. Search by cues serves clip hoards in film and news work. The work strips spare parts from the tool build.
Catalog record
| CREATOR | Yuqian Zheng, Mariana-Iuliana Georgescu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16582
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
X-Aligner : Composed Visual Retrieval without the Bells and Whistles Yuqian Zheng, Mariana-Iuliana Georgescu Abstract Composed Video Retrieval (CoVR) facilitates video retrieval by combining visual and textual queries. However, existing CoVR frameworks typically fuse multimodal inputs in a single stage, achieving only marginal gains over initial baseline. To address this, we propose a novel CoVR framework that leverages the representational power of Vision-Language Models (VLMs). Our framework incorporates a novel cross-attention module X-Aligner , composed of cross-attention layers that progressively fuse visual and textual inputs and align their multimodal representation with that of the target video. To further enhance the representation of the multimodal query, we incorporate the caption of the visual query as an additional input. The framework is trained in two stages to preserve the pretrained VLM representation. In the first stage, only the newly introduced module is trained, while in the second stage, the textual query encoder is also fine-tuned. We implement our framework on top of BLIP-family architecture, namely BLIP and BLIP-2, and train it on the Webvid-CoVR data set. In addition to in-domain evaluation on Webvid-CoVR-Test, we perform zero-shot evaluations on the Composed Image Retrieval (CIR) data sets CIRCO and Fashion-IQ. Our framework achieves state-of-the-art performance on CoVR obtaining a Recall@1 of 63.93% on Webvid-CoVR-Test, and demonstrates strong zero-shot generalization on CIR tasks. 1 Introduction The rapid expansion of media content demands for more sophisticated retrieval systems capable of handling complex search queries, such as multimodal ones. Traditional text- or image-based retrieval methods lack the ability to retrieve content involving compositional changes. Therefore, Composed Video Retrieval (CoVR) and Composed Image Retrieval (CIR) have become crucial tasks, enabling users to find specific visual content by combining a reference image or video with a natural language modification query. The goal of a composed visual retrieval is to identify the visual target that best matches the visual query (image or video) modified by the corresponding text query. Several state-of-the-art approaches [ 9 , 13 ] propose training-free techniques that leverage Large Language Models (LLMs) or large vision models for zero-shot composed visual retrieval. However, these methods often underperform because they lack explicit training on how to precisely transform a visual query based on a textual query. Other works [ 21 , 23 , 20 ] adapt Vision-Language Models (VLMs) to composed visual retrieval by fine-tuning them on CIR or CoVR data sets. Figure 1 : We propose a novel framework that leverages multi-stage cross-attention and video captions to accurately align multimodal queries. As shown, the single-stage baseline CoVR-BLIP [ 23 ] fails interpret the “turn it into a male” instruction correctly, retrieving a video with a male subject. In contrast, our method fuses the visual, textual, and caption inputs to retrieve the correct video of a man painting. Our approach achieves state-of-the-art performance, such as a Recall@1 of 63.93% on Webvid-CoVR-Test. For example, Ventura et al . [ 23 ] freeze the vision encoder of BLIP [ 12 ] and fine-tune its multimodal encoder. However, performance gains on the CoVR benchmarks have remained limited, with subsequent state-of-the-art models only marginally outperforming the baselines proposed by Ventura et al . [ 23 ] . We observe that information from the input query is processed only once by the model encoders, without being integrated across multiple stages. Single-stage input integration limits the models’ ability to iteratively refine and align multimodal representations, potentially leading to suboptimal understanding of complex queries, where subtle textual modifications require deeper interaction with the visual features. In this work, we address the Composed Video Retrieval task by leveraging the rich knowledge encoded in VLMs. We propose X-Aligner , a module that combines visual and textual inputs through multiple cross-attention layers to improve the alignment between their representations. To further enhance the representation, the input embeddings are integrated across multiple stages. The caption of the visual query, automatically generated by InternVL-G [ 4 ] , is also incorporated to enrich semantic understanding by providing high-level visual context in language space that complements the visual features. To better adapt VLMs to the CoVR task, we design a novel two-stage training framework. In the first stage, we exclusively train the newly added cross-attention components ( X-Aligner ). In the second stage, we jointly fine-tune the textual query encoder along with X-Aligner , while keeping the remaining model parameters frozen to prevent catastrophic forgetting of the pretrained VLM’s original multimodal knowledge. The proposed framework is built upon two widely used VLMs, namely BLIP [ 12 ] and BLIP-2 [ 11 ] . Its effectiveness is validated through extensive experiments on the Webvid-CoVR data set [ 23 ] , where it achieves a state-of-the-art Recall@1 of 63.93% on the test set. As illustrated in Figure 1 , our framework successfully combines multimodal inputs to retrieve the target video, showcasing its ability to handle complex compositional queries. Furthermore, the strong generalization of the model is demonstrated through zero-shot CIR evaluations on the FashionIQ [ 25 ] and CIRCO [ 2 ] data sets. We summarize our contributions as follows. • We introduce a simple, yet powerful framework that aligns visual and textual inputs through progressive cross-modal reasoning, effectively adapting pretrained VLMs for Composed Video Retrieval. • We achieve state-of-the-art performance on the WebVid-CoVR dataset, surpassing existing baselines by a significant margin. • We demonstrate that the representations learned by our framework generalize well to Composed Image Retrieval, achieving competitive performance in zero-shot CIR task. 2 Related Work Composed Image Retrieval (CIR): There has been a growing interest in CIR [ 24 ] in the research community in recent years. Several CIR methods [ 3 , 18 , 1 , 5 , 9 , 6 , 28 , 29 ] leverage CLIP [ 17 ] to encode the query (reference) image and query text into a shared embedding space for target image retrieval. Specifically, Gu et al . [ 6 ] introduced a language-only training framework for CIR that maps images to text representations, thereby reformulating CIR as a text-to-image retrieval task. Several other works have proposed training-free methods [ 9 , 26 , 19 , 13 , 15 ] . For instance, Karthik et al . [ 9 ] proposed CIReVL, a framework that utilizes LLMs to generate a target image caption by combining the text query and the visual input caption, thus performing text-to-image retrieval. On the other hand, Li et al . [ 13 ] approached the CIR task from an image-to-image retrieval perspective, employing an image generation model to synthesize proxy images based on multimodal queries. To further improve CIR performance, Levy et al . [ 10 ] introduced the Large-Scale Composed Image Retrieval (LaSCo) dataset and proposed a new CIR model based on BLIP [ 12 ] . While the aforementioned methods are specifically designed for CIR, however, our work presents a framework that is fine-tuned on a video data set and demonstrates its generalization capabilities through performing Zero-Shot (ZS) Composed Image Retrieval. Composed Video Retrieval (CoVR): The Composed Video Retrieval task was introduced by Ventura et al . [ 23 ] , who presented the first CoVR framework by adapting BLIP [ 12 ] and training it on WebVid-CoVR data set, integrating multimodal inputs via summation or