Benchmarking Composed Image Retrieval for Applied Earth Observation

Bill Psomas, Dionysis Christopoulos, Thanasis Petropoulos, Nikos Efthymiadis, Ioannis Kakogeorgiou, Ondřej Chum, Yannis Avrithis, Giorgos Tolias, Konstantinos Karantzalos · 2026

Read Benchmarking Composed Image Retrieval for Applied Earth Observation by Bill Psomas, Dionysis Christopoulos, Thanasis Petropoulos, Nikos Efthymiadis, Ioannis Kakogeorgiou, Ondřej Chum, Yannis Avrithis, Giorgos Tolias, Konstantinos Karantzalos (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24442.txt.

Introduction

Psomas and eight co-authors benchmark mixed image search for Earth viewing, pairing a source image with a text note. How modern methods transfer to Earth viewing pictures remains little known. The team adapts and tests sample methods with six vision-language bases on PatternCom. They also add xView2-CIR, a change-centric set for harm watch. Training-free methods give strong marks, while change-centric search poses new hurdles because scene look must be preserved. The study sets a handy benchmark for mixed search in archive browsing.

Catalog record

CREATORBill Psomas, Dionysis Christopoulos, Thanasis Petropoulos, Nikos Efthymiadis, Ioannis Kakogeorgiou, Ondřej Chum, Yannis Avrithis, Giorgos Tolias, Konstantinos Karantzalos
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77243
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24442

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Benchmarking Composed Image Retrieval for Applied Earth Observation Bill Psomas, Dionysis Christopoulos, Thanasis Petropoulos, Nikos Efthymiadis, Ioannis Kakogeorgiou, Ondřej Chum, Yannis Avrithis, Giorgos Tolias, Konstantinos Karantzalos Abstract Remote sensing composed image retrieval (RSCIR) enables search in large satellite image archives using composed queries that combine a reference image with a textual modifier. Although RSCIR offers a flexible interface for expressing targeted retrieval intent, the transferability of modern composition methods to Earth observation (EO) imagery and their relevance to operational EO workflows remain underexplored. We address this gap through a unified benchmark and an application-oriented study. First, we systematically adapt and evaluate representative composed image retrieval methods with six vision-language backbones on PatternCom under a standardized protocol, analyzing their behavior across backbones, composition strategies, and query types. Second, we introduce xView2-CIR, a change-centric dataset for disaster and damage monitoring, where retrieval is conditioned on scene identity and a target post-event state. Our results show that training-free composition methods provide strong and scalable baselines for EO retrieval, while change-centric retrieval presents different challenges from attribute-based retrieval, particularly due to the need to preserve scene identity. Overall, this study establishes a practical benchmark for RSCIR and positions composed retrieval as a complementary tool for remote sensing image retrieval, archive exploration, and change analysis. [orcid=0000-0001-5381-0312] keywords Remote Sensing Image Retrieval ,Composed Image Retrieval ,Multimodal Retrieval ,Vision–Language Models ,Earth Observation ,Benchmarking 1 Introduction The explosive growth of earth observation (EO) data has enabled large-scale monitoring of the planet, but has also made it increasingly difficult for users to navigate massive satellite image archives and retrieve content matching specific information needs. Remote sensing image retrieval (RSIR) [ 1 ] addresses this challenge by searching EO archives from a query, most commonly an image. Prior RSIR research spans unisource and cross-source [ 110 ] settings, as well as single-label [ 30 , 80 , 14 ] and multi-label [ 37 , 34 , 75 ] formulations. Despite this progress, a fundamental limitation persists: most RSIR systems are driven by a single modality query. A user must typically choose between an image query or a text query, which restricts expressivity when the retrieval intent includes targeted modifications beyond overall similarity. Figure 1: Composed Image Retrieval for applied Earth Observation. We illustrate how composed queries ( query image + query text ) enable controllable retrieval by specifying a targeted change. (a) Activity monitoring: a parking-lot query image composed with being empty retrieves visually similar parking areas with low vehicle occupancy. (b) Capacity monitoring: A facility query image composed with having four retrieves visually similar sites exhibiting a different quantity attribute. (c) Disaster monitoring: a pre-event query image composed with post-fire retrieves post-event imagery of the same scene consistent with wildfire impact. (d) Urban development: a pre-construction query image composed with new buildings retrieves the same scene exhibiting recent construction. For each query, retrieved candidates (ranked left-to-right, top-to-bottom) are produced by FreeDom [ 19 ] with OpenAI CLIP [ 67 ] ; green borders indicate correct retrievals under the respective evaluation criterion (a,b: same class + target attribute value, c,d: same scene + target change), while red borders indicate mismatches. Images in (a,b) are from PatternNet [ 112 ] , (c) from xView2 [ 41 ] , and (d) from LEVIR-CC [ 50 ] . Many operational remote sensing workflows are inherently compositional . For example, an analyst may wish to retrieve locations visually similar to a reference region but with a different attribute, such as a parking lot with lower occupancy or a facility with a larger number of storage tanks, as shown in Figure 1 (a,b). In change-centric applications, the goal may instead be to retrieve imagery of the same location under a different state, such as after a wildfire or recent construction, as shown in Figure 1 (c,d). Such intent is difficult to express with unimodal queries alone. Remote sensing composed image retrieval (RSCIR) [ 64 ] addresses this limitation by combining a reference image with a natural-language modifier, aiming to retrieve images that preserve the relevant visual context while satisfying the textual specification [ 71 , 19 , 65 ] . Despite its promise, RSCIR remains insufficiently studied from an applied EO perspective. Existing work [ 64 ] introduced the task, but it remains unclear how modern composed image retrieval methods behave under a unified evaluation protocol, how well they transfer across general-purpose and remote-sensing vision–language backbones, and whether composed retrieval remains effective when relevance depends on scene identity and a target post-event state rather than class-level attribute matching. These questions are important for assessing whether RSCIR can support practical EO workflows such as archive exploration, infrastructure monitoring, disaster response, and urban change analysis. This work advances RSCIR through a unified benchmark and an application-oriented study. We first benchmark representative composition methods on PatternCom [ 64 ] , an attribute-modification testbed derived from PatternNet [ 112 ] . We evaluate six vision–language backbones, including CLIP [ 67 ] , SigLIP [ 105 ] , RemoteCLIP [ 51 ] , and SkyCLIP [ 94 ] , and adapt representative composition methods such as Pic2Word [ 71 ] , SEARLE [ 2 ] , FreeDom [ 19 ] , MagicLens [ 106 ] , and BASIC [ 65 ] to EO imagery. Beyond aggregate retrieval metrics, we analyze method behavior across backbones, query types, and composition strategies to provide practical guidance for selecting RSCIR pipelines. To connect benchmarking with operational EO needs, we further introduce xView2-CIR, a new dataset derived from xView2 [ 41 ] for change-centric composed retrieval in disaster and damage monitoring. In xView2-CIR, a query consists of a pre-event reference image and a textual modifier such as post-fire , and the goal is to retrieve the corresponding post-event image of the same location. This setting evaluates whether composed retrieval can support location-aware and semantically steerable search, complementing established change-centric workflows such as change detection, damage assessment, and rapid mapping. In summary, we make the following contributions: 1. We establish a unified benchmark for RSCIR on PatternCom, with domain-grounded adaptations of representative composition methods and a standardized protocol spanning six vision-language backbones. 2. We introduce xView2-CIR, a new evaluation dataset for change-centric composed retrieval in disaster and damage monitoring, where relevance depends on both scene identity and target post-event state. 3. We provide empirical analysis and practical guidance showing when composed retrieval is useful for remote sensing applications, and how its behavior differs between attribute-based and change-centric settings. 2 Related Work Remote Sensing Image Retrieval. With the aim to effectively search and retrieve information from extensive remote sensing (RS) image archives, remote sensing image retrieval (RSIR) can be categorized into unisource and cross-source [ 110 ] . Initially, RSIR methods focus on handcrafted and low-level visual features [ 58 , 56 , 63 , 45 , 5 , 88 , 74 , 89 , 9 , 17 ] . With the advent of deep learning, neural networks are utilized for unisource single-label retrieval: (a) as feature extractors [ 47 , 31 , 6 , 100 , 59 , 22 , 83 , 33 , 70 , 30 ] , (b) trained from scratch [ 111 , 107 , 113 , 54 , 92 , 80 ,

Read the complete text