A Data-Centric Approach to Pedestrian Attribute Recognition: Synthetic Augmentation via Prompt-driven Diffusion Models
Alejandro Alonso, Sawaiz A. Chaudhry, Juan C. SanMiguel, Álvaro García-Martín, Pablo Ayuso-Albizu, Pablo Carballeira · 2025
Read A Data-Centric Approach to Pedestrian Attribute Recognition: Synthetic Augmentation via Prompt-driven Diffusion Models by Alejandro Alonso, Sawaiz A. Chaudhry, Juan C. SanMiguel, Álvaro García-Martín, Pablo Ayuso-Albizu, Pablo Carballeira (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02099.txt.
Introduction
Alejandro Alonso leads five co-authors in a push to lift trait reads on street shots by fixing training data, not the model. The task reads traits like garb and gear from shots of folks on foot. Rare traits lag most, since data sets hold few of them and models learn them poorly. Their data first route starts by finding which traits read weakly in each set.
A prompt led pipeline that makes images from noise then makes fake shots of folks on foot. The fake shots keep the look and rules of the real data sets. A merge plan folds fake shots into training with prompt based tags and a tweaked loss. Rare trait reads rise, and whole model scores rise past the aimed traits. New scene skill grows with no change to the model build.
Catalog record
| CREATOR | Alejandro Alonso, Sawaiz A. Chaudhry, Juan C. SanMiguel, Álvaro García-Martín, Pablo Ayuso-Albizu, Pablo Carballeira |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 116 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-02099
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Data-Centric Approach to Pedestrian Attribute Recognition: Synthetic Augmentation via Prompt-driven Diffusion Models Alejandro Alonso, Sawaiz A. Chaudhry, Juan C. SanMiguel, Álvaro García-Martín, Pablo Ayuso-Albizu, Pablo Carballeira Abstract Pedestrian Attribute Recognition (PAR) is a challenging task as models are required to generalize across numerous attributes in real-world data. Traditional approaches focus on complex methods, yet recognition performance is often constrained by training dataset limitations, particularly the under-representation of certain attributes. In this paper, we propose a data-centric approach to improve PAR by synthetic data augmentation guided by textual descriptions. First, we define a protocol to identify weakly recognized attributes across multiple datasets. Second, we propose a prompt-driven pipeline that leverages diffusion models to generate synthetic pedestrian images while preserving the consistency of PAR datasets. Finally, we derive a strategy to seamlessly incorporate synthetic samples into training data, which considers prompt-based annotation rules and modifies the loss function. Results on popular PAR datasets demonstrate that our approach not only boosts recognition of underrepresented attributes but also improves overall model performance beyond the targeted attributes. Notably, this approach strengthens zero-shot generalization without requiring architectural changes of the model, presenting an efficient and scalable solution to improve the recognition of attributes of pedestrians in the real world. 1 Introduction Pedestrian Attribute Recognition (PAR) aims to identify multiple soft biometric attributes of individuals in human-centric AI systems. Despite significant advances in deep learning-based approaches, achieving stable and consistent performance across all attributes remains a challenge [ 1 ] . Most state-of-the-art models struggle with imbalanced and under-represented attribute distributions derived from real-world data [ 2 ] , leading to suboptimal recognition for certain attributes, even with sophisticated learning architectures. Many attributes appear infrequently or with limited variability, making it difficult for models to learn generalizable representations. Additionally, in zero-shot learning scenarios, where models must recognize attributes of people identities that were not seen during training, further expose the limitations of conventional PAR approaches. Recently, generative AI has emerged as a solution to address data scarcity in computer vision tasks [ 3 ] . Methods based on Generative Adversarial Networks, Variational Autoencoders, and Diffusion Models (DMs) have been extensively explored for synthesizing high-quality images to support data augmentation and zero-shot learning. Among these, DMs have recently gained prominence to generate highly realistic and diverse images while providing fine-grained control over content [ 4 ] . Despite the success of DMs, their potential for augmenting PAR datasets remains relatively unexplored, presenting a promising avenue for improving attribute recognition performance. In this paper, we propose a data-centric approach to improve PAR performance through synthetic data. Instead of increasing model complexity, we enhance data representation quality by generating synthetic pedestrian images to address the imbalance of under-represented attributes. Extensive experiments on popular datasets (RAPv1, RAPv2 [ 5 ] , and RAPzs [ 1 ] ) show that our method boosts recognition of augmented attributes while preserving overall robustness. Our contributions are threefold: (1) A systematic protocol to identify weakly recognized attributes across multiple datasets; (2) A prompt-driven pipeline leveraging diffusion models to generate synthetic pedestrian images while preserving dataset consistency; and (3) An integration strategy for PAR data augmentation that modifies the loss function to seamlessly incorporate synthetic samples into training, ensuring smooth adaptation of new data. 2 Related Work 2.1 PAR: Methods and Datasets Recent approaches for Pedestrian Attribute Recognition (PAR) integrate attribute-specific characteristics beyond architectural improvements [ 7 ] , where ResNet is a popular backbone [ 1 , 8 , 9 ] . For example, discrimination can be enhanced by introducing location-aware associations of attributes with body regions [ 10 ] . Similarly, attention mechanisms [ 11 ] can be used to model attribute correlations, enabling to refine predictions. Dataset biases can also introduce inconsistencies for learning these attribute relationships, being addressed by information-theoretic regularization [ 12 ] . Multi-task transformers have been also proposed to capture global and local relations [ 13 ] . In summary, many recent approaches focus on algorithmic complexity or inter-attribute correlations to enhance PAR performance. These complex PAR approaches rely on datasets with challenges for data distribution and generalization. For example, PETA [ 14 ] and PA100K [ 15 ] cover both indoor and outdoor environments, while RAPv1 and its extension RAPv2 [ 5 ] focus on indoor surveillance with a richer set of attributes. While efforts such as PETAZs and RAPZs [ 1 ] have aimed to address zero-shot learning, PAR datasets remain highly imbalanced, with certain attributes severely underrepresented. Consequently, models trained on them struggle to generalize, and leveraging inter-attribute relationships proves ineffective, leading to biased learning. 2.2 Text-guided Synthetic Image Generation Generative AI has become a powerful tool to mitigate dataset biases and augment training visual data [ 16 ] , with Generative Adversarial Networks (GANs) widely used to enhance data diversity. Early attempts to generate synthetic PAR data relied on unconditional GANs [ 17 ] , lacking fine-grained control over individual attributes. More recently, Diffusion Models (DMs) have surpassed GANs [ 4 ] , achieving high realism and diversity while allowing conditioning mechanisms, such as prompts or images. This flexibility has led to the rise of highly popular models, including DALL·E [ 18 ] and Stable Diffusion [ 19 ] . Hence, DMs arise as an option for generating PAR-like images with precise control over attributes, ensuring dataset consistency while adapting to diverse real-world scenarios. Generating visual data through textual prompts is a powerful yet non-trivial task, especially when aiming to match the appearance of a target domain. DMs allow text-to-image generation (txt2img), but standard prompting can lead to inconsistent attributes, unrealistic data, or domain mismatches [ 20 ] . Therefore, Prompt Engineering refines prompt design to better align generated images with the target domain without modifying the model. For example, Wildcards and Template-Based Prompts introduce controlled attribute variation ensuring dataset realism [ 21 ] , while Textual Inversion learns custom token embeddings to generate domain-consistent data without retraining [ 22 ] . Conversely, Model Adaptation enhances control in txt2img diffusion. Prompt-to-Prompt [ 23 ] refines attributes via attention map adjustments, while LoRA fine-tuning [ 24 ] efficiently adapts models by updating a small parameter subset. Despite PAR’s challenges with class imbalance and domain bias, these techniques remain unexplored. This paper first examines prompt engineering to ensure realistic, attribute-consistent images before model adaptation. 3 Proposed Approach Our proposal considers three steps to enhance Pedestrian Attribute Recognition (PAR) with synthetic images. First, we identify weakly recognized attributes by analyzing class imbalance and recognition performance. Next, we generate synthetic pedestrian images using a diffusion-based pipeline, ensuring realistic attribute representations. Finally, we integrate the synthetic data into training by labeling the data and adapting the loss function. The next subsections outline the proposal; further details of the proposal are available in the supplementary material 1 1 1 http://www-vpu.eps.uam.es/AVSS2025PAR . 3.1 Identifying Weakly Recognized Attributes We start by identifying the worst-performing attributes to apply our data-centric synthetic augmentation. Given one or more PAR training/test sets D^{j}=\{(x^{j}_{i},y^{j}_{i}),i=1,2,\ldots,N_{j}\} , where j indexes each dataset that contains N_{j} pedestrian images. Each image is annotated with M shared binary attributes y^{j}_{i}\in\{0,1\}^{M} . Our goal is to determine which attributes require augmentation. Identification Criteria: We identify weakly recognized attributes using three criteria: (1) class