Improving Open-World Object Localization by Discovering Background
Ashish Singh, Michael J. Jones, Kuan-Chuan Peng, Anoop Cherian, Moitreya Chatterjee, Erik Learned-Miller · 2025
Read Improving Open-World Object Localization by Discovering Background by Ashish Singh, Michael J. Jones, Kuan-Chuan Peng, Anoop Cherian, Moitreya Chatterjee, Erik Learned-Miller (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17626.txt.
Introduction
Ashish Singh and five colleagues improve open-world object localization by teaching a network what background looks like. The task is to find all objects in an image, including classes never seen during training. Most methods focus on characterizing objects themselves, through new objectives or auxiliary signals like depth.
Their framework instead discovers background regions, defined as redundant areas with low information content. An object proposal network is then trained to propose nothing inside those regions. Experiments on standard benchmarks show significant gains over previous state-of-the-art methods. The authors argue that knowing where objects are not is a powerful guide to objectness.
Catalog record
| CREATOR | Ashish Singh, Michael J. Jones, Kuan-Chuan Peng, Anoop Cherian, Moitreya Chatterjee, Erik Learned-Miller |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-17626
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving Open-World Object Localization by Discovering Background Ashish Singh, Michael J. Jones, Kuan-Chuan Peng, Anoop Cherian, Moitreya Chatterjee, Erik Learned-Miller Abstract Our work addresses the problem of learning to localize objects in an open-world setting, i.e . , given the bounding box information of a limited number of object classes during training, the goal is to localize all objects, belonging to both the training and unseen classes in an image, during inference. Towards this end, recent work in this area has focused on improving the characterization of objects either explicitly by proposing new objective functions (localization quality) or implicitly using object-centric auxiliary-information, such as depth information, pixel/region affinity map etc . In this work, we address this problem by incorporating background information to guide the learning of the notion of objectness. Specifically, we propose a novel framework to discover background regions in an image and train an object proposal network to not detect any objects in these regions. We formulate the background discovery task as that of identifying image regions that are not discriminative, i.e . , those that are redundant and constitute low information content. We conduct experiments on standard benchmarks to showcase the effectiveness of our proposed approach and observe significant improvements over the previous state-of-the-art approaches for this task. 1 Introduction Figure 1 : An example of a training image showing a ground truth annotated object (green box), unlabeled objects that are not in the known training classes (purple boxes), and clear non-object/background regions (red boxes) that are automatically classified as non-object regions by BOWL ( ours ) and used in training. Open-world object localization (OWOL) refers to the task of localizing all objects, including novel object instances in a given image by learning from a limited set of predefined object classes. Unlike the traditional closed-world object localization setting, where the task is to localize object classes only seen during training, OWOL is a more realistic and challenging problem that evaluates generalization performance of the learned model to novel object classes. Learning to localize object instances, not seen during training time, is important for many downstream applications like autonomous agents, robotics, and self-driving cars. As a result, there has been a growing interest in developing methods for object detection/localization in an open-world setting over the last few years Kim et al. 2022 ; Wang et al. 2022 . Previous works in OWOL have primarily focused on improving objectness learning. Specifically, Kim et al . Kim et al. 2022 proposed incorporating localization-quality as a loss function to learn a more generalizable object representation, whereas, Wang et al . Wang et al. 2022 explicitly enriches the training data by synthesizing pseudo-annotations for unannotated regions. While the above approaches show improvement in localizing unseen object classes, their performance is still limited. Specifically, we observe that during inference, either: (1) novel objects with features like shape, color, textures, etc . not observed in known classes are not detected, or (2) non-object regions are erroneously detected as objects due to noisy pseudo-annotations during model training. These observations allude to the challenging issue of developing and learning a generalizable notion of objectness. Learning the objectness of a region is non-trivial and most assumptions developed to characterize objectness do not generalize to unseen environments. In this work, we tackle this challenge by exploring a complementary source of information, i.e . , non-objectness of a region. We show that identifying non-object regions and incorporating them during the learning process significantly improves the localization of unseen object classes and the generalization to novel domains on various challenging benchmarks. In particular, we propose to improve open-world object localization by incorporating non-objectness information during the learning process as illustrated in Figure 1 . We first model the non-objectness information by constructing a codebook comprised of embeddings of exemplar non-object patches, extracted from a pre-trained model. This codebook is then used to identify seed anchors representing non-object regions while training the object localization method. This provides additional self-supervision in training the localization method along with labeled information from known classes. Our evaluations show that by incorporating our non-object model into the object localization pipeline, our proposed method, Background-aware Open-World Localizer ( BOWL ), significantly improves over previous methods on multiple benchmarks for OWOL. Specifically, we surpass the previous baselines by 4.0% on the AR@100 measure for detecting non-VOC classes when the training data consist of 20 PASCAL-VOC Everingham et al. 2010 classes. Furthermore, we show a domain generalization improvement of 2.0% on the AR@100 measure for open-world localization in the wild, when evaluating a model trained with the COCO dataset Lin et al. 2014 on the ADE20K dataset Zhou et al. 2019 . Going forward, we sometimes use the term, ”background”, in an informal way to mean non-object regions. Overall, in this work, we make the following contributions: 1. We propose BOWL , a novel, open-world object localization method based on background-awareness. 2. We formalize the notion of non-objectness of image regions and propose a novel, unsupervised method for discovering non-object regions with high precision. 3. Our extensive evaluation shows that incorporating non-objectness information in BOWL leads to improved recall in detecting unseen objects, outperforming competing methods for this task. 2 Related Work In this section, we discuss related prior works and delineate the differences from our proposed approach. Open-World Object Localization addresses the problem of localizing objects in images under the open-world setting. Early methods in object localization Alexe et al. 2012 ; Arbeláez et al. 2014 ; Endres and Hoiem 2010 ; Manen et al. 2013 ; Zitnick and Dollár 2014 adopted a bottom-up approach to creating a relatively small set of candidate bounding boxes that covered the objects in an image. These methods utilized hand-crafted heuristics based on raw image information like color, edge, and pixel-contrast to characterize objectness Uijlings et al. 2013 ; Zitnick and Dollár 2014 . In contrast, learning based methods Ren 2015 ; Kuo et al. 2015 ; Li et al. 2019 ; O Pinheiro et al. 2015 ; Wang et al. 2019 adopt a top-down approach, where the model is trained end-to-end with objectness defined by a set of training categories in the dataset. These learning-based methods show considerable improvement over heuristics-based approaches in terms of precision and recall when localizing object classes from the training set. However, they struggle to localize novel object categories. In particular, these methods incorrectly assume any unannotated region of a training image to be a non-object region and thus suppress detection of novel objects at inference time. To resolve this issue, Kim et al . Kim et al. 2022 proposed to modify Faster-RCNN Ren 2015 by replacing classification heads with class-agnostic objectness heads which are trained using base-class annotations, thus avoiding any incorrect assumptions about background. Saito et al . Saito et al. 2022 proposed to instead crop labeled object regions in training images and paste them into images not containing any unlabeled objects to avoid the faulty assumption that unlabeled regions of training images do not contain any objects. While the above methods focus on mitigating the background bias, another family of methods have focused on improving object localization recall by pseudo-annotating novel objects. In particular, Wang et al . Wang et al. 2022