OmniPerson: Unified Identity-Preserving Pedestrian Generation

Changxiao Ma, Chao Yuan, Xincheng Shi, Yuzhuo Ma, Yongfei Zhang, Longkun Zhou, Yujia Zhang, Shangze Li, Yifan Xu · 2025

Read OmniPerson: Unified Identity-Preserving Pedestrian Generation by Changxiao Ma, Chao Yuan, Xincheng Shi, Yuzhuo Ma, Yongfei Zhang, Longkun Zhou, Yujia Zhang, Shangze Li, Yifan Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-02554.txt.

Introduction

Changxiao Ma and eight co-authors generate fake pedestrian images to train person recognition systems. Person re-identification matches the same person across cameras, but large labeled datasets are hard to build. Privacy limits and labeling costs keep real data scarce.

Synthetic people could fill the gap, yet past generators often lose the person's identity or give users little control. OmniPerson is a unified generator built to preserve identity while staying controllable. Better synthetic data could improve recognition training without filming more real people, a gain for both accuracy and privacy. Fake crowds can be made in bulk, while real crowds take time, cost, and care to film on the street.

Catalog record

CREATORChangxiao Ma, Chao Yuan, Xincheng Shi, Yuzhuo Ma, Yongfei Zhang, Longkun Zhou, Yujia Zhang, Shangze Li, Yifan Xu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-02554

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OmniPerson: Unified Identity-Preserving Pedestrian Generation Changxiao Ma, Chao Yuan, Xincheng Shi, Yuzhuo Ma, Yongfei Zhang, Longkun Zhou, Yujia Zhang, Shangze Li, Yifan Xu Abstract Person re-identification (ReID) suffers from a lack of large-scale high-quality training data due to challenges in data privacy and annotation costs. While previous approaches have explored pedestrian generation for data augmentation, they often fail to ensure identity consistency and suffer from insufficient controllability, thereby limiting their effectiveness in dataset augmentation. To address this, We introduce OmniPerson , the first unified identity-preserving pedestrian generation pipeline for visible/infrared image/video ReID tasks. Our contributions are threefold: ➀ We proposed OmniPerson , a unified generation model , offering holistic and fine-grained control over all key pedestrian attributes. Supporting RGB/IR modality image/video generation with any number of reference images, two kinds of person poses, and text. Also including RGB-to-IR transfer and image super-resolution abilities. ➁ We designed Multi-Refer Fuser for robust identity preservation with any number of reference images as input, making OmniPerson could distill a unified identity from a set of multi-view reference images, ensuring our generated pedestrians achieve high-fidelity pedestrian generation. ➂ We introduce PersonSyn , the first large-scale dataset for multi-reference, controllable pedestrian generation, and present its automated curation pipeline which transforms public, ID-only ReID benchmarks into a richly annotated resource with the dense, multi-modal supervision required for this task. Experimental results demonstrate that OmniPerson achieves SoTA in pedestrian generation, excelling in both visual fidelity and identity consistency. Furthermore, augmenting existing datasets with our generated data consistently improves the performance of ReID models. We will open-source the full codebase, pretrained model, and the PersonSyn dataset. Figure 1 : The Muti-tasks generation results of OmniPerson : Controllable Pedestrian Generation from Diverse References. 1 Introduction Person re-identification (ReID) aims to retrieve and track target individuals across non-overlapping camera views based on appearance features (e.g. clothing and body shape). In recent years, the field has been dominated by deep learning models 1 that have achieved remarkable performance on established benchmarks 23 ; 16 . The success of these data-driven methods is, however, critically dependent on the availability of large-scale, diverse, and meticulously annotated datasets. This reliance on massive datasets constitutes a major real-world bottleneck, fraught with challenges including prohibitive annotation costs, stringent data privacy regulations, and the difficulty of capturing the long tail of variations in appearance, pose, and environmental conditions 37 . This data scarcity severely limits the robustness and generalizability of even the SoTA ReID models in real-world deployments 9 , motivating the exploration of alternative data sources. Generative data augmentation has emerged as a powerful strategy to overcome the challenges of data scarcity and limited diversity in ReID. The value of this approach is well-established, as extensive research has demonstrated that augmenting training sets with high-quality synthetic data significantly enhances both the discriminative power 33 and generalization capability 57 of ReID models. The technology enabling this has progressed rapidly, from early methods using classical image processing 59 and virtual engines 57 , to the first wave of generative approaches based on GANs 43 . More recently, the advent of large-scale diffusion models, such as Stable Diffusion 38 , has marked a paradigm shift in image synthesis. Capitalizing on their unprecedented realism and control 10 , the latest works now focus on applying these diffusion-based models specifically for high-fidelity person synthesis 29 ; 8 . However, despite the strong generative capabilities of these state-of-the-art person generation models, applying them directly to the specialized task of ReID data augmentation reveals two critical limitations. (1) First, they exhibit a critical failure in high-fidelity identity preservation , especially when generating subjects under drastic variations in pose and viewpoint. The core challenge in ReID data augmentation is creating these novel views, but this is precisely where existing methods, which typically rely on a single reference image for appearance guidance 29 ; 8 ; 50 , are fundamentally flawed. As illustrated in Figure 2 , a single view provides an incomplete and often biased representation, failing to capture details visible only from other angles (e.g., the back of a shirt or a logo on a sleeve). Consequently, when tasked with rendering the person from a novel perspective, the model is compelled to hallucinate missing visual information, resulting in significant inconsistencies and ”identity drift.” This introduces conflicting signals into the ReID training data; instead of learning a robust identity signature, the ReID model is fed ambiguous information that can corrupt its feature space and degrade performance. Figure 2 : Exiting works vs OmniPerson. (2) Second, they lack a unified and disentangled control mechanism. Existing methods operate in silos: text-driven approaches struggle to faithfully condition on a specific visual identity, often leading to inconsistent results. 40 Conversely, methods conditioned on a visual reference are often pose-driven, offering little to no explicit control over crucial attributes like background or modality. This fragmented control scheme prevents the targeted synthesis of challenging, long-tail scenarios (e.g., a specific person in a rare pose against a novel background, captured in the infrared spectrum) that are crucial for systematically bolstering the robustness of ReID models. To overcome these fundamental limitations of identity drift and fragmented control, we introduce OmniPerson , a unified generation pipeline designed specifically for identity-preserving pedestrian data synthesis, and PersonSyn , a detailed reconstructed dataset for pedestrian generation. Our main contributions are four-fold: • A Unified Identity-Preserving Pedestrian Generation model: OmniPerson. Supporting RGB/IR modality image/video generation with any number of reference images, two kinds of person poses, and text. Also including RGB-to-IR transfer and image super-resolution abilities. • Multi-Refer Fuser for OmniPerson. It achieve the robust identity preservation with any number of reference images as input, making OmniPerson could distill a unified identity from a set of multi-view reference images, ensuring our generated pedestrians achieve high-fidelity pedestrian generation. • PersonSyn Dataset. To facilitate research in multi-reference, controllable pedestrian generation, we reorganized the public pedestrian datasets and obtained PersonSyn dataset with the comprehensive annotations for pedestrian generation, including 2D/3D poses, disentangled background plates, classified viewpoints, textual attributes, and reference image sets. • Extensive experiments demonstrate that OmniPerson achieves SOTA performance in both generation quality and identity consistency. Furthermore, we validate on downstream tasks and achieve SOTA performance. 2 Related Works 2.1 Person Re-Identification Person re-identification has become a cornerstone task in computer vision, aiming to associate pedestrian identities across non-overlapping camera views and varying time instances. Recent advances in ReID have been largely driven by deep learning, with convolutional neural networks (CNNs) 20 and vision transformers (ViTs) 1 proving particularly effective in boosting recognition accuracy. State-of-the-art approaches typically integrate feature extraction with metric learning frameworks 26 ; 21 ; 42 ; 25 ; 52 ; 49 ; 53 ; 51 , enabling the learning of discriminative representations that remain robust to challenges such as pose variations, viewpoint changes, and illumination differences. Standard datasets such as Market-1501 58 have been widely adopted to evaluate ReID algorithms under controlled conditions, while SYSU-MM01 46 focuses on matching identities between visible and infrared images. Although existing methods demonstrate strong benchmark performance on these datasets, the limited number of images and identities, along with homogeneity of collection scenarios, result in ReID models lacking robustness in real-world deployments. 2.2 Data Augmentation for ReID To address data

Read the complete text