Improving Zero-Shot Object-Level Change Detection by Incorporating Visual Correspondence
Hung H. Nguyen, Pooyan Rahmanzadehgervi, Long Mai, Anh Totti Nguyen · 2025
Read Improving Zero-Shot Object-Level Change Detection by Incorporating Visual Correspondence by Hung H. Nguyen, Pooyan Rahmanzadehgervi, Long Mai, Anh Totti Nguyen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05555.txt.
Introduction
Hung Nguyen and co-authors improve change detection between two photos in a computing paper from 2025. The task is to spot objects added or removed, even when the two shots come from distinct angles. It matters for inspection work and camera surveillance.
Past methods had three gaps: no tests on pairs with no change, no pinpointing of where a change sits, and weak skill on new scenes. The new method uses matched regions before and after a change. Those matches guide training and also cut false alarms at test time. Accuracy rose by a wide margin over past work. The approach also held up better on scenes it had never seen.
Catalog record
| CREATOR | Hung H. Nguyen, Pooyan Rahmanzadehgervi, Long Mai, Anh Totti Nguyen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-05555
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving Zero-Shot Object-Level Change Detection by Incorporating Visual Correspondence Hung H. Nguyen, Pooyan Rahmanzadehgervi, Long Mai, Anh Totti Nguyen Abstract Detecting object-level changes between two images across possibly different views ( Fig. 1 ) is a core task in many applications that involve visual inspection or camera surveillance. Existing change-detection approaches suffer from three major limitations: (1) lack of evaluation on image pairs that contain no changes, leading to unreported false positive rates; (2) lack of correspondences ( i.e . , localizing the regions before and after a change); and (3) poor zero-shot generalization across different domains. To address these issues, we introduce a novel method that leverages change correspondences (a) during training to improve change detection accuracy, and (b) at test time, to minimize false positives. That is, we harness the supervision labels of where an object is added or removed to supervise change detectors, improving their accuracy over previous work [ 25 ] by a large margin. Our work is also the first to predict correspondences between pairs of detected changes using estimated homography and the Hungarian algorithm. Our model demonstrates superior performance over existing methods, achieving state-of-the-art results in change detection and change correspondence accuracy across both in-distribution and zero-shot benchmarks. (a) COCO-Inpainted (b) STD (c) Kubric-Change (d) Synthtext (e) OpenImages-Inpainted CYWS [ 25 ] Ours Figure 1 : At an optimal confidence threshold, CYWS [ 25 ] (top row) sometimes still produces false positives— \Box in (a) & (c)—and fails to detect changes (a). Dashed - - - boxes show groundtruth changes. First, we encourage detectors to be more aware of changes via a novel contrastive loss. Second, our Hungarian-based post-processing reduces false positives (a), improves change-detection accuracy (b), and estimates correspondences (c–d), i.e., paired changes such as ( \Box , \Box ) and ( \Box , \Box ). Our work (bottom row) is the first to estimate change correspondences compared to prior works [ 25 , 26 , 40 ] (top row). More qualitative results in ( Fig. A4 ) . 1 Introduction Identifying key changes between two images is a core task that powers many applications [ 25 ] , e.g . , to detect changes across brain scans [ 6 , 22 ] , a missing car in a parking lot ( Fig. 1 ) [ 31 , 13 ] , or a defective product in a manufacturing pipeline [ 36 ] . However, existing work has three major limitations. First, most papers did not test on pairs of images where there are no changes [ 25 , 26 ] and therefore do not measure false positives. Many image-difference captioning benchmarks contain only change cases [ 34 ] or only a small subset of no-change examples, e.g., 10% of Spot-the-Diff [ 13 ] . Second, prior models are trained to detect only changes; yet, such detected changes are not too usable in the downstream application when there are many changes predicted per image but no correspondence provided ( Fig. 1 ). Third, many image-difference prediction works are specialized for a single domain (e.g., remote sensing [ 37 ] ) and do not measure zero-shot generalization to unseen datasets [ 37 , 40 ] . In this paper, we address these three problems by leveraging change correspondences during (a) training to improve change detection precision, and (b) test time to reduce false positives. Compared to prior works [ 25 , 26 , 40 ] , our work is the first to predict correspondences in addition to the changes ( Fig. 1 ). That is, we propose a post-processing algorithm based on an estimated homography and the Hungarian algorithm [ 17 , 2 ] to reduce false positives ( Fig. 1 ) of a state-of-the-art change detector [ 25 ] . Intuitively, first, we run a pre-trained change detector [ 25 ] on a pair of images to collect a set of predicted changes on each image. Then, we project the predicted boxes in image 1 onto image 2 and filter out those that do not substantially overlap with any predicted changes in image 2, arriving at higher-precision change predictions ( Fig. 4 ). A similar procedure is used to filter out the predicted changes in image 2. We then harness the Hungarian algorithm to predict correspondences in addition to the predicted changes ( Fig. 1 ). Our main findings are: 1 1 1 Code and data are available on github . 1. Leveraging correspondence labels in finetuning detectors leads to state-of-the-art change detectors, outperforming CYWS [ 25 ] by a large margin (from +1.05 to +9.04 in mAP) on all five benchmarks ( Section 5.1 ). 2. Our proposed contrastive matching loss function for finetuning change detectors also improves the accuracy in predicting correspondences, from +1.31 to +6.56 ), on all five benchmarks ( Section 5.2 ). 3. Moreover, we have established a new metric for evaluating matching scores among different models, facilitating a consistent and comparative assessment of change detection performance ( Section 2.3 ). 4. We present OpenImages-Inpainted, i.e . , a novel change detection dataset with \sim 1.3M image pairs, where image pairs consist of exactly 1 change derived from realistic scenes of the OpenImages [ 18 ] dataset. Our OpenImages-Inpainted has no view-transformation artifacts (compared to COCO-Inpainted [ 25 ] ) and minimal inpainting artifacts (compared to Img-Diff [ 14 ] ). 2 Problem formulation 2.1 Definition of Changes We define a change to be an addition, absence, or modification of an object in one image compared to the other (see Fig. 2 a–b). A major challenge is to detect such object-level changes in the presence of changes in camera viewpoint (COCO-I, KC), colors (COCO-I), or lighting (STD), which we do not aim to detect. The objects that change include humans, animals (COCO-I), man-made objects (COCO-I, STD, KC, OI), and letters (SC). In the case where the same object moves from one location to another ( Fig. 2 c) across two images, we expect two changes to be detected: (1) An object is removed from the first location in image 1, and (2) an object is added to the second location in image 2. That is, two pairs of corresponding changes are to be predicted. (a) viewpoint, colors (b) viewpoint (c) move = remove + add Figure 2 : Example predicted changes . In , our method detects a removal of an elephant (a) despite viewpoint and color differences between two images. A tiny, white object (b) removal is detected despite viewpoint differences in . In , a blue truck moving from one location to another is correctly detected as two changes: a removal and an addition . Colored, solid-bordered boxes, e.g . , ( \Box , \Box ), ( \Box , \Box ), ( \Box , \Box ), show predicted paired changes. 2.2 Five benchmarks Following [ 25 ] , we train and test both our model and CYWS [ 25 ] on COCO-Inpainted. Additionally, we test these models zero-shot on four unseen change-detection benchmarks: STD [ 13 ] , Kubric-Change [ 25 ] , and Synthtext-Change [ 25 ] and our proposed OpenImages-Inpainted. COCO-Inpainted (COCO-I) [ 25 ] contains 57K, 3K, and 4.5K image pairs in the train,