Towards Unbiased Source-Free Object Detection via Vision Foundation Models Journal: Pattern Recognition

Zhi Cai, Yingjie Gao, Yanan Zhang, Xinzhu Ma, Di Huang · 2026

Read Towards Unbiased Source-Free Object Detection via Vision Foundation Models Journal: Pattern Recognition by Zhi Cai, Yingjie Gao, Yanan Zhang, Xinzhu Ma, Di Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12765.txt.

Introduction

Zhi Cai and four co authors tackle object detection across domains without source data. Source free object detection adapts a model to a new domain without revisiting source images. That setting fits privacy and storage limits in real deployments. Existing methods suffer from source bias, though: the adapted model stays skewed toward the source domain.

That skew leads to poor generalization on the target domain. It also causes error accumulation during self training, as bad labels reinforce themselves. The team use vision foundation models toward unbiased adaptation. Their work was prepared for a pattern recognition journal. Reliable source free detection would ease deployment of vision systems in new places.

Catalog record

CREATORZhi Cai, Yingjie Gao, Yanan Zhang, Xinzhu Ma, Di Huang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-12765

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Unbiased Source-Free Object Detection via Vision Foundation Models Journal: Pattern Recognition Zhi Cai, Yingjie Gao, Yanan Zhang, Xinzhu Ma, Di Huang Abstract Source-Free Object Detection (SFOD) has garnered much attention in recent years by eliminating the need of source-domain data in cross-domain tasks, but existing SFOD methods suffer from the Source Bias problem, i.e. the adapted model remains skewed towards the source domain, leading to poor generalization and error accumulation during self-training. To overcome this challenge, we propose Debiased Source-free Object Detection (DSOD), a novel VFM-assisted SFOD framework that can effectively mitigate source bias with the help of powerful VFMs. Specifically, we propose Unified Feature Injection (UFI) module that integrates VFM features into the CNN backbone through Simple-Scale Extension (SSE) and Domain-aware Adaptive Weighting (DAAW). Then, we propose Semantic-aware Feature Regularization (SAFR) that constrains feature learning to prevent overfitting to source domain characteristics. Furthermore, we propose a VFM-free variant, termed DSOD-distill for computation-restricted scenarios through a novel Dual-Teacher distillation scheme. Extensive experiments on multiple benchmarks demonstrate that DSOD outperforms state-of-the-art SFOD methods, achieving 48.1% AP on Normal-to-Foggy weather adaptation, 39.3% AP on Cross-scene adaptation, and 61.4% AP on Synthetic-to-Real adaptation. 1 Introduction Unsupervised Domain Adaptive Object Detection (UDAOD) [ 36 , 21 , 48 , 50 ] has attracted significant research attention in recent years, primarily driven by the critical challenge of domain shift in real-life applications. The domain shift problem arises when there is a distribution discrepancy between the training (source domain) and testing (target domain) data, leading to substantial performance degradation. However, a fundamental limitation of UDAOD approaches is their dependence on source domain data during adaptation, which introduces additional practical constraints regarding data storage, privacy, and computational requirements. In contrast, Source-Free Object Detection (SFOD) [ 28 , 42 , 10 , 24 , 46 ] represents a more practical and challenging paradigm that eliminates the need for source data during adaptation. Specifically, SFOD methods [ 42 , 10 , 24 , 46 ] operate under the constraint that only a source-pretrained model and unlabeled target data are available. Compared to UDAOD methods [ 36 , 21 , 48 , 50 ] , SFOD approaches are more prone to the Source Bias problem (model are biased towards source rather than target domain), as illustrated in Fig. 1 (I,II,III). One key reason is that UDAOD methods can learn domain-invariant features [ 52 ] by leveraging both source and target domain data during training, which facilitates effective knowledge transfer from the source domain to the target domain. In contrast, SFOD relies solely on unlabeled target data, which restricts the model’s ability to capture robust cross-domain representations and leads to suboptimal knowledge transfer. Consequently, SFOD models often exhibit poorer generalization performance and are more susceptible to error accumulation throughout the self-training process. The recent emergence of Vision Foundation Models (VFMs), such as DINOv2 [ 34 ] , offers a promising direction to address the mentioned challenges. We argue that integrating VFMs into SFOD provides two key advantages: Semantic Complementarity and Regularization Effect . First, VFMs offer generic and robust visual representations, while source-pretrained models provide instance-level and class-aware features, enabling complementary representation learning. Second, the frozen VFM can serve as a regularizer, preventing the model from overfitting to narrow source-domain patterns. By leveraging more diverse and unbiased information, this approach mitigates source bias and enhances generalization performance. Figure 1: Conceptual comparison of different domain adaptation paradigms. (I) Ideal Adaptation: Achieves complete knowledge transfer from the source domain ( red ) to the target domain ( blue ). (II) Traditional SFOD: Prone to source bias, as indicated by the predominance of red circles representing source-domain characteristics. (III) UDAOD: Enables more effective adaptation but requires access to source-domain data. (IV) VFM-assisted SFOD: Mitigates source bias by incorporating unbiased knowledge ( purple ) while eliminating the need for source-domain data access. In this paper, we propose a novel method named Debiased Source-Free Object Detection (DSOD) that mitigates source bias by leveraging the capability of VFMs, as depicted in Fig. 1 (IV). It adopts a dual-tower architecture that comprises a DINOv2 [ 34 ] encoder and a standard ResNet [ 17 ] encoder to achieve less biased adaptation. To enrich the output features with unbiased knowledge, we introduce an Unbiased Feature Injection (UFI) mechanism, which enhances the semantic richness of original CNN features by integrating features extracted from DINOv2. Then we propose a Semantic-Aware Feature Regularization (SAFR) module to strengthen the intrinsic correlation between the CNN and VFM encoders, thereby further eliminating feature-level bias. Besides, to alleviate training instability, we design a Domain-Aware Adaptive Weighting (DAAW) strategy that adaptively determines fusion weights via a heuristic approach and employs gradual weighting to smooth optimization conflicts. To facilitate the deployment in the edge devices, we further introduce a Dual-Teacher (DT) distillation scheme. It helps distill the knowledge from DSOD to a VFM-free variant, named DSOD-Distill, that incurs no extra inference burden . To summarize, our contributions are five-fold: 1. We present DSOD, a novel framework that significantly enhances SFOD performance by exploiting VFMs for complementary semantics and regularization. 2. We design two core modules, UFI and SAFR. The former injects semantic-rich features of VFM into the detection model, and the latter restricts the manifold learning of CNN encoders by semantic-aware regularization from VFMs. 3. We further introduce a DAAW strategy to automate fusion weight selection and stabilize the training with smooth weighting schedule. 4. We devise a DT method to transfer the VFM’s knowledge into a compact, VFM-free model (DSOD-Distill), enabling practical deployment. 5. Extensive experiments on multiple benchmarks demonstrate the effectiveness of the proposed method. 2 Related Works 2.1 Object Detection The remarkable success of deep neural networks in image classification [ 17 ] has spurred the development of CNN-based detectors, which can be broadly categorized into two-stage [ 35 ] , one-stage [ 30 ] and anchor-free methods [ 39 , 54 ] . As a representative two-stage approach, Faster R-CNN [ 35 ] employs a Region Proposal Network (RPN) to generate region proposals, which are subsequently refined by an ROI head. While effective, this framework suffers from inherent limitations in efficiency and generalization due to its multi-stage design and reliance on hand-crafted anchor boxes. In contrast, single-stage methods [ 32 , 30 ] integrate proposal generation and detection into a single forward pass, achieving superior computational efficiency and simplicity. Anchor-free methods advance this direction further by predicting objects directly from key points such as the center [ 39 ] or corners [ 26 ] . Despite these advancements, most CNN-based detectors still produce multiple predictions per object and rely on the non-maximum suppression (NMS) strategy to remove redundant detections. More recently, the proposal of DETR [ 3 ] reformulates the pipeline of detection to a set-prediction problem with the introduce of transformer [ 40 ] . Designed with an end-to-end paradigm, DETR totally eliminates the need of hand-crafted components such as NMS and anchor boxes. However, it suffers from slow-convergence issue. Following work [ 55 ] explores utilizing sparse attention in transformer layer and achieves impressive progress. Some works [ 49 , 22 ] suppose the slow-convergence issue is also due to the one-to-one matching and improve DETR with query-denoising strategy [ 49

Read the complete text