Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification

Jiachen Li, Xiaojin GongCollege of Information Science and Electronic EngineeringZhejiang University, Dongping ZhangCollege of Information EngineeringChina Jiliang University · 2026

Read Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification by Jiachen Li, Xiaojin GongCollege of Information Science and Electronic EngineeringZhejiang University, Dongping ZhangCollege of Information EngineeringChina Jiliang University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14012.txt.

Introduction

Li, Gong, and Zhang improve person re-identification across unseen domains with vision-language alignment at several grains. Models train on some camera domains and must work on new ones. Plain vision-language models give only global features, which miss fine identity cues. Their CLIP-based method writes prompts for body parts and aligns them with matching image regions. An attention module pulls out part features, guided by pseudo labels from a grounding expert. Tests under single- and multi-source protocols beat prior methods.

Catalog record

CREATORJiachen Li, Xiaojin GongCollege of Information Science and Electronic EngineeringZhejiang University, Dongping ZhangCollege of Information EngineeringChina Jiliang University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71094
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14012

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification Jiachen Li, Xiaojin GongCollege of Information Science and Electronic EngineeringZhejiang University, Dongping ZhangCollege of Information EngineeringChina Jiliang University Abstract Domain Generalized person Re-identification (DG Re-ID) is a challenging task, where models are trained on source domains but tested on unseen target domains. Although previous pure vision-based models have achieved significant progress, the performance remains further improved. Recently, Vision-Language Models (VLMs) present outstanding generalization capabilities in various visual applications. However, directly adapting a VLM to Re-ID shows limited generalization improvement. This is because the VLM only produces with global features that are insensitive to ID nuances. To tacle this problem, we propose a CLIP-based multi-grained vision-language alignment framework in this work. Specifically, several multi-grained prompts are introduced in language modality to describe different body parts and align with their counterparts in vision modality. To obtain fine-grained visual information, an adaptively masked multi-head self-attention module is employed to precisely extract specific part features. To train the proposed module, an MLLM-based visual grounding expert is employed to automatically generate pseudo labels of body parts for supervision. Extensive experiments conducted on both single- and multi-source generalization protocols demonstrate the superior performance of our approach. The implementation code will be released at https://github.com/RikoLi/MUVA . 1 Introduction Person Re-identification (Re-ID) 42 ; 24 ; 25 has been extensively studied to match identities across multiple non-overlapped cameras. In despite of considerable advances, conventional Re-ID models degenerate when tested in new scenarios due to domain gaps. Consequently, Domain Generalized Person Re-identification (DG Re-ID) has been proposed, which learns models on source domains and is tested on unseen target domains. Over few past years, various pure vision-based paradigms have been proposed to tackle DG Re-ID, including feature disentanglement and normalization 29 ; 83 ; 100 , mixture-of-experts 72 ; 15 ; 21 , meta-learning 14 ; 87 ; 47 ; 82 and so on. Although significant progress has been achieved, DG Re-ID remains challenging to apply in real-world scenarios. This is because pure visual information contains limited generalization knowledge. In an image, domain-specific information ( e.g . illumination, view angle, image resolution and so on) tends to tightly entangle with ID-specific information. When models are trained, the ID recognition learning is accompanied by the learning of domain-specific information. Such models may perform well on test samples from source domains, but they degenerate drastically when transferred to quite different scenarios due to significant gaps of vision data distribution. Figure 1 : Comparison of the single- and multi-grained alignments. (a) indicates the single-grained alignment. Similar IDs from diverse visual domains can be summarized by a sentence in a global view. Domain-specific attributes such as illumination, view angle, image resolution and so on are eliminated due to the nature of language. However, the single-grained description ignores ID nuances. (b) indicates our multi-grained alignment, where extra fine-grained descriptions of parts are employed to represent more subtle ID differences. Note that exact words in descriptions are used as examples to explain our idea. In implementation, several learnable prompts are adopted for more flexible descriptions instead of manually designed words. Recently, Vision-Language Models (VLMs) 50 ; 27 perform remarkably well on out-of-distribution data due to the utilization of language modality in vision tasks. Inspired by this, we expect that the potential of DG Re-ID can be further excavated if the VLM is adopted to align vision and language modalities. As illustrated in Figure 1 (a), each ID can be summarized by a textual description. The description is inherently ID-relevant. Domain-specific attributes are naturally eliminated when each image is aligned with text. But some methods 68 ; 69 ; 84 still obtain suboptimal performances even if they have included text in training. This is because they merely exploit single-grained information. Although text is generalizable, it is simultaneously challenged to represent the nuances among similar IDs. Details like hairstyles, clothes patterns and so on are not sufficiently represented in single-grained manner. However, these detailed cues are crucial for precise recognition, which have been extensively utilized in many previous Re-ID tasks 58 ; 60 ; 34 ; 98 ; 97 ; 19 ; 79 . Therefore, a multi-grained alignment between two modalities is expected as illustrated in Figure 1 (b), which additionally focuses on local details of body parts and describes the ID nuances comprehensively. To complete multi-grained alignment, it is important to extract local features at different granularities. Most previous approaches adopt stripe-like partition 58 ; 60 ; 34 to equally divide an image into stripes along height direction. This strategy is simple and easy to implement. However, the perception field of each rigid stripe is constrained to a fixed size, making it unable to flexibly capture local features on deformable body parts, especially when a person is walking or varying body poses. This tends to damage the vision-language alignment of nuanced semantics. A better approach to adaptively extract local features is required to mitigate this problem. We propose a MUlti-grained Vision-language Alignment (MUVA) framework by adapting a well-known VLM model CLIP 50 to solve aforementioned issues within a unified architecture. To address the nuance perception deficiency, we design a multi-grained vision-language prompt learning strategy. Specifically, we propose learnable multi-grained prompts, which are particularly designed to align both the entire human body and its local parts. To sufficiently adapt CLIP 50 , a two-stage training pipeline is adopted. Initially, the learnable prompts of each ID are trained to align with corresponding images, obtaining multi-grained textual descriptions. After that, the descriptions offer a cross-modal constraint during Re-ID training to learn generalizable features. To align with textual descriptions, local visual features at multiple granularities are extracted. Instead of using rigid stripes, we propose an Adaptively Masked Multi-head Self-Attention (AM-MSA) module to improve CLIP 50 ’s image encoder for precise part feature extraction. In each module, a transformer is equipped with a bypass branch containing a Residual Mask Prediction (RMP) module and a Mask Gating (MG) module. The RMP module predicts specific part masks layer by layer with cross-attention mechanism. The MG module further employs masked attention on the transformer with the predicted masks to focus on detailed semantics. To train the AM-MSA module, a Visual Grounding Expert (VGE) based on a Multi-modal Large Language Model (MLLM) 41 ; 40 is used to automatically generate pseudo labels for each part, thereby avoiding tedious manual annotation. In inference, the AM-MSA module can adaptively predict body parts, enabling our model to accurately capture both global context and fine-grained local features. In summary, our major contributions are as follows: • We adapt vision-language alignment to improve DG Re-ID with the inherent generalization capability of language. To cope with the nuance perception deficiency in single-grained alignment, we propose a multi-grained vision-language alignment framework based on a multi-grained prompt learning strategy. • To precisely extract local information, we design an adaptively masked multi-head self-attention module to predict part regions. To train this module without tedious manual annotation, an MLLM-based visual grounding expert is employed to generate pseudo labels for different parts as supervision. • Extensive experiments conducted under both single- and multi-source DG Re-ID protocols demonstrate the effectiveness of our approach, surpassing state-of-the-art methods by a remarkable margin. 2 Related Works 2.1 Domain Generalized and Part-Based Person

Read the complete text