Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Pooling
Hojun Son, Asma Almutairi, Arpan Kusari · 2025
Read Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Pooling by Hojun Son, Asma Almutairi, Arpan Kusari (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18446.txt.
Introduction
Hojun Son, Asma Almutairi, and Arpan Kusari attack a hidden flaw in object detectors: they learn backgrounds along with objects. A detector trained on cars in streets may falter when the same cars appear elsewhere, because it leaned on the street as a clue. The authors trace this context bias to the pooling step in convolutional networks, which blends object and background signals together.
Their fix, Mask Pooling, feeds the network a mask of the object region and pools foreground and background apart, forcing the model to judge the object itself. They also build a harsh benchmark that pastes objects onto fully random backgrounds as an ultimate test of robustness. Detectors trained with the new pooling held up better when the domain shifted. The pair of contributions offers both a reason the bias arises and a practical way to train it away.
Catalog record
| CREATOR | Hojun Son, Asma Almutairi, Arpan Kusari |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18446
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Pooling Hojun Son, Asma Almutairi, Arpan Kusari Abstract Context bias refers to the association between the foreground objects and background during the object detection training process. Various methods have been proposed to minimize the context bias when applying the trained model to an unseen domain, known as domain adaptation for object detection (DAOD). But a principled approach to understand why the context bias occurs and how to remove it has been missing. In this work, we provide a causal view of the context bias, pointing towards the pooling operation in the convolution network architecture as the possible source of this bias. We present an alternative, Mask Pooling, which uses an additional input of foreground masks, to separate the pooling process in the respective foreground and background regions and show that this process leads the trained model to detect objects in a more robust manner under different domains. We also provide a benchmark designed to create an ultimate test for DAOD, using foregrounds in the presence of absolute random backgrounds, to analyze the robustness of the intended trained models. Through these experiments, we hope to provide a principled approach for minimizing context bias under domain shift. 1 Introduction Domain adaptation for object detection (DAOD) addresses performance degradation caused by domain shifts between training and testing datasets. Despite advances in object detector architectures [ 20 , 32 , 55 , 42 , 65 , 46 ] and training strategies [ 48 , 69 , 17 , 60 , 85 , 66 ] , models still struggle to generalize to unseen domains [ 71 , 31 , 72 ] . Manual annotation on new datasets is expensive and labor-intensive. To alleviate this, sim-to-real approaches leverage synthetic data for training and real-world evaluation [ 67 , 22 , 28 ] . However, the performance gap—known as the sim-to-real gap—remains substantial, and no method has surpassed the “ORACLE" model trained on fully labeled target data [ 30 ] . This is largely due to spurious factors and distributional perturbations learned during training [ 74 , 75 ] . DAOD algorithms mitigate domain shifts through feature alignment [ 14 , 6 , 7 , 84 , 19 ] , adversarial domain classifiers [ 15 , 75 ] , knowledge distillation [ 30 , 53 , 3 , 4 ] , and hybrid methods [ 2 , 11 , 78 , 23 ] . While these techniques aim to extract target-domain priors, these techniques still regenerate context bias between foreground (FG) and background (BG) during adaptation [ 62 ] . Causal tools can provide a natural way to measure the effect of the association between the FG and BG. In particular, Structural Causal Models (SCMs) provide a principled framework for capturing associational, interventional, and counterfactual causality [ 59 , 37 , 33 , 82 , 29 , 68 ] , which traditional explainable AI (xAI) methods are not equipped to handle [ 47 , 5 , 50 , 49 ] . Ultimately, modeling interventions to find causal discovery supports the development of predictive models that are robust to distributional shifts commonly encountered in real-world settings but also enables that can be extended to support reasoning. Based on this knowledge, several recent studies have applied causal interventions to DAOD [ 83 , 25 , 56 , 27 , 44 , 38 , 76 , 8 , 40 , 61 ] , with varying interpretations of context. One approach treats context as a causal factor of the image and performs intervention using joint representations and attention mechanisms [ 25 ] . Visual causality meaning frequent existence pairs in images and awareness of challenging group were modeled by causal intervention [ 27 ] . Other study addresses contrast and spatial distribution biases as confounding factors in adaptation [ 44 ] , or identify and mitigate target discrimination bias in teacher-student frameworks using conditional causal intervention [ 38 ] . Spurious correlations have also been targeted through frequency-domain augmentation and multi-view adversarial discriminators to isolate domain-invariant causal features [ 76 ] . Additionally, ambiguous boundaries are resolved by modeling a category-causality chain and intervening with saliency-based causal maps [ 8 ] . The SIGMA framework [ 40 ] reformulates DAOD as a semantic-complete graph matching problem to address mismatched semantics and intra-class variance. While not a formal application of causal intervention (e.g., using do-calculus or SCMs), SIGMA adopts causal reasoning principles to reduce spurious correlations caused by missing semantics or noisy BGs. However, it remains uncertain whether the performance gains are derived from the underlying causal hypotheses or simply due to increased model complexity, such as the addition of more parameters or layers. In particular, none of the previous research have taken a critical look at the pooling operation in Convolutional Neural Network (CNN) architectures which combines features for a particular layer without any prior discrimination of FG and BG features. While various efficient pooling methods have been proposed to better capture contextual information during training [ 35 , 77 , 80 , 79 , 45 , 63 , 16 , 64 ] , these approaches often abstract low-level features into high-level representations that may inadvertently encode contextual biases as semantic cues. In particular, max pooling indiscriminately selects the highest activation within a kernel, regardless of its spatial origin, which can introduce FG-BG bias and degrade performance [ 77 ] . These facts raise a critical question: can commonly used pooling operations in CNNs introduce artifacts that confound learning and contribute to domain shift? Despite extensive research on alternative pooling techniques, their causal implications remain unexplored. We hypothesize that FG and BG features, being inherently independent, may form spurious associations during training, ultimately impairing generalization to novel domains. To investigate this, we designed experiments leveraging object detection tasks to examine the causal relationship between FG-BG associations and model performance. We applied causal intervention-based analysis using ground-truth FG masks to isolate effects and draw causal inferences. We focused on multi objects detection because it provides a balanced complexity—more intricate than image classification, yet more manageable than semantic segmentation—making it suitable for causal analysis. Our findings offer insights for developing models with stronger domain generalization and out-of-distribution (OOD) robustness. We hope that our findings and experimental outcomes can provide remarkable perspectives on DAOD community. Our main contributions are as follows: • We demonstrate that pooling operations can introduce causal artifacts that negatively affect model performance for DAOD task. • We propose an alternate pooling strategy, Mask Pooling, to remove the causal effects of FG-BG associations on object detection outcomes and showcase the robustness empirically in different source and target domains using multiple neural network models. • We present a benchmark to evaluate model robustness for DAOD task by creating synthesized images with known FGs juxtaposed on random BGs. Through these contributions, we aim to provide a novel perspective to the DAOD community to mitigate context-bias using source-only domain. The paper is arranged as follows: Section 2 defines the problem in causal terms, section 3 provides the pooling, models, datasets, evaluation and perturbation, section 4 provides the experimental results and section 5 provides the discussion. 2 Problem definition The left panel of Fig.