Beyond Frequency: Scoring-Driven Debiasing for Object Detection via Blueprint-Prompted Image Synthesis
Xinhao Cai, Liulei Li, Gensheng Pei, Tao Chen, Jinshan Pan, Yazhou Yao, Wenguan Wang · 2025
Read Beyond Frequency: Scoring-Driven Debiasing for Object Detection via Blueprint-Prompted Image Synthesis by Xinhao Cai, Liulei Li, Gensheng Pei, Tao Chen, Jinshan Pan, Yazhou Yao, Wenguan Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18229.txt.
Introduction
Xinhao Cai, with Liulei Li and five co-authors, tackles bias in object detection in a 2025 paper. Object detection means teaching computers to find and label things in pictures. Detectors often do worse on rare or unusual objects, and adding fake training pictures can keep the same bias.
The team scores how well each group is represented, not just how often it appears. They then draw a clear visual blueprint, meaning a precise layout picture, to guide new image creation. Tests show the gap shrinks for underrepresented groups, with clear gains for large and rare objects. The approach offers a fairer way to train detectors.
Catalog record
| CREATOR | Xinhao Cai, Liulei Li, Gensheng Pei, Tao Chen, Jinshan Pan, Yazhou Yao, Wenguan Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18229
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Frequency: Scoring-Driven Debiasing for Object Detection via Blueprint-Prompted Image Synthesis Xinhao Cai, Liulei Li, Gensheng Pei, Tao Chen, Jinshan Pan, Yazhou Yao, Wenguan Wang Abstract This paper presents a generation-based debiasing framework for object detection. Prior debiasing methods are often limited by the representation diversity of samples, while naive generative augmentation often preserves the biases it aims to solve. Moreover, our analysis reveals that simply generating more data for rare classes is suboptimal due to two core issues: i) instance frequency is an incomplete proxy for the true data needs of a model, and ii) current layout-to-image synthesis lacks the fidelity and control to generate high-quality, complex scenes. To overcome this, we introduce the representation score (RS) to diagnose representational gaps beyond mere frequency, guiding the creation of new, unbiased layouts. To ensure high-quality synthesis, we replace ambiguous text prompts with a precise visual blueprint and employ a generative alignment strategy, which fosters communication between the detector and generator. Our method significantly narrows the performance gap for underrepresented object groups, e . g ., improving large/rare instances by 4.4/3.6 mAP over the baseline, and surpassing prior L2I synthesis models by 15.9 mAP for layout accuracy in generated images. 1 Introduction The reliability of object detection models is fundamentally limited by biases in their training data, manifesting as skewed distributions across object categories ( Ouyang et al. 2016 ) , sizes ( Herranz et al. 2016 ) , and spatial locations ( Zheng et al. 2024 ) . Conventional debiasing strategies, such as resampling ( Cui et al. 2019 ) or re-weighting ( Tan et al. 2020 ) , attempt to mitigate this by adjusting the influence of training instances based on frequency. While effective to a degree, these methods are constrained by the visual vocabulary of the original dataset. They can re-balance the influence of rare samples but cannot generate novel appearances or contexts to fill representational gaps. Generation-based data augmentation ( Wu et al. 2023 ; Trabucco et al. 2024 ) has emerged as a promising alternative to overcome this limitation. By synthesizing entirely new training samples, these methods hold the potential to create a more balanced dataset. However, current solutions for object detection typically follow a layout-to-image (L2I) synthesis pipeline ( Chen et al. 2024a ; Wang et al. 2024 ) , where the layouts used as conditions for data generation are directly sampled from the original training set. Thus the generation process inevitably preserves the very biased distributions they aim to solve, leaving a clear need for a truly bias-aware generation strategy. But what would an effective generation-based debiasing framework entail? Our investigation in § 2 reveals that: i) simply combining the frequency-centric debiasing view with generative approaches, i . e ., generating more images for rare data groups, is not the final answer. It can outperform both traditional augmentation techniques ( e . g ., copy-paste, random flip, crop) and bias-agnostic L2I synthesis, yet still falls short of the gains achieved by enriching the training set with more real samples; ii) the quality of samples generated by current L2I synthesis methods remains below that of real data, as models trained on synthetic samples consistently underperform those trained on real ones. The problems can be two sets: ❶ Instance frequency is an incomplete proxy to determine the most needed data of a model ( Chawla et al. 2002 ; He & Garcia 2009 ) . According to the controlled experiments in § 2 , we find that certain high-performing and data-rich groups ( e . g ., large objects) can be more ‘data hungry’ and gain greater benefit from additional data compared to low-performing groups with limited samples ( e . g ., small objects). Relying solely on frequency can result in suboptimal interventions. ❷ Even with a perfect, bias-targeted layout and a powerful generation model, current L2I approaches struggle to render new samples faithfully. Prior L2I methods primarily focus on fusing layout conditions into the generation process, with limited attention given to enhancing the fidelity of generated images to real-world data. Moreover, these methods directly translate 2D spatial arrangements into 1D text sequences. This introduces ambiguity and lacks the fine-grained control for complex scenes with specific object relationships and occlusion ( Johnson et al. 2018 ) . In this work, we propose a targeted debiasing framework that automatically diagnoses the underrepresented data groups and executes precise generation to diversify training data. To tackle ❶, we introduce a representation score (RS) that moves beyond simple frequency counts to quantify how well a concept is represented across both sample density and representation diversity. The RS then guides a bias-aware recalibration module which constructs new, unbiased layouts to fill the identified representational gaps. Furthermore, the entire diagnosis-then-create pipeline is embedded within a dynamic debiasing engine that leverages detector errors to continuously refine the RS, ensuring the system remains adaptive and focused on the challenging biases throughout training. To tackle ❷, we replace ambiguous text prompts with a visual blueprint , i . e ., canvases composed of colored rectangles that specify the class, size, and position of each object. This provides the generative model with direct and unambiguous instructions on object relationships, occlusion, and instance identity, ensuring the precise synthesis of debiased samples. Next, we exploit the duality between L2I synthesis and object detection, where the output of one task naturally serves as the input to the other. On this basis, we form a generative alignment mechanism that enforces consistency within an “Image-Layout-Image” loop. This facilitates communication between the generator and detector by penalizing the detector when it produces layouts that are insufficient for faithful image synthesis. Unlike frequency-based methods, our RS-driven debiasing strategy tackles limited sample diversity by completing the truly underrepresented data groups with samples featuring novel appearances; moving beyond conventional generative augmentation, visual-blueprint and generative alignment facilitates precise synthesis of high-quality data targeting specific representation gaps. Consequently, our method demonstrates strong debiasing effectiveness. It establishes a new SOTA and greatly narrows the performance gap for underrepresented groups, e . g ., + 3.6 mAP for rare classes, + 3.2 mAP for instances at image borders, + 4.4 and 1.9 mAP for large and small objects on MS COCO. Our approach also demonstrates high generation fidelity, with the accuracy of layouts in synthesized images surpassing prior SOTA by 15.9 mAP, when compared against existing L2I synthesis models. 2 The Frequency Trap and Fidelity Gap: A Motivating Study In this section, we conduct controlled experiments across three dimensions: spatial location, category frequency, and object size, to assess the influence of different data augmentation and debiasing strategies. All studies utilize Faster R-CNN ( Ren et al. 2015 ) with a ResNet-50 ( He et al. 2016 ) backbone. Hyperparameters are kept identical across models. We first train the detector on a random 1/4 subset of the MS COCO training set. We then measure the mAP by enriching the 1/4 subset by factors of 4/3, 2, and 4 with: i) resampling ( Gupta et al. 2019 ) rare data groups via standard data augmentation techniques like copy-paste, random flip, and crop (termed Data Aug ); ii) bias-agnostic L2I synthesis to generate new samples using layouts from training sets (termed