RSRCC : A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of- N N Ranking

Roie Kazoom, Yotam Gigi, George Leifman · 2026

Read RSRCC : A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of- N N Ranking by Roie Kazoom, Yotam Gigi, George Leifman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20623.txt.

Introduction

Kazoom and four others present RSRCC, a benchmark for change questions with 126k questions: 87k training, 17.1k checks, and 22k test. Unlike sets that describe image gaps, RSRCC asks local, single-change questions needing thought about one meaning change. It is built with a guided sorting method. Possible regions come from meaning splitting masks, are screened by a vector model, and are checked through vision-language sorting with Best-of-N ranking as the doubt stage. That method filters noisy and unclear items at scale while keeping real changes.

Catalog record

CREATORRoie Kazoom, Yotam Gigi, George Leifman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83114
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20623

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RSRCC : A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of- N N Ranking Roie Kazoom, Yotam Gigi, George Leifman Abstract Traditional change detection identifies where changes occur, but does not explain what changed in natural language. Existing remote sensing change captioning datasets typically describe overall image-level differences, leaving fine-grained localized semantic reasoning largely unexplored. To close this gap, we present RSRCC , a new benchmark for remote sensing change question-answering containing 126k questions, split into 87 ​ k 87k training, 17.1 ​ k 17.1k validation, and 22 ​ k 22k test instances. Unlike prior datasets, RSRCC is built around localized, change-specific questions that require reasoning about a particular semantic change. To the best of our knowledge, this is the first remote sensing change question-answering benchmark designed explicitly for such fine-grained reasoning-based supervision. To construct RSRCC , we introduce a hierarchical semi-supervised curation pipeline that uses Best-of- N N ranking as a critical final ambiguity-resolution stage. First, candidate change regions are extracted from semantic segmentation masks, then initially screened using an image-text embedding model, and finally validated through retrieval-augmented vision-language curation with Best-of- N N ranking. This process enables scalable filtering of noisy and ambiguous candidates while preserving semantically meaningful changes. The dataset is available at https://huggingface.co/datasets/google/RSRCC . Figure 1: Example samples from our generated dataset. Each sample contains a pair of before and after satellite images, followed by one or more visual localized questions. The questions are designed to capture semantic changes, such as new construction, demolition, vegetation loss, or no visible change. The correct answers are highlighted in green within the figure. 1 Introduction Figure 2: Pipeline overview. On the left (1), semantic mask differences are analyzed using Intersection over Union (IoU) and connected component analysis to localize candidate changes, denoted by \Delta x . In the middle (2), an image-text encoder processes cropped regions x\in\mathbb{R}^{H\times W} and retrieves the top- k most similar candidates \{x_{1},x_{2},\ldots,x_{k}\} for preliminary semantic validation. On the right (3), ambiguous cases are resolved through retrieval-augmented Best-of- N ranking. A large language model scores the query against retrieved annotated examples, assigning a relevance score r\in\{1,..,n\} that promotes semantically consistent matches and filters low-confidence candidates. Change detection (CD) from satellite imagery aims to identify differences between multi-temporal scenes, supporting applications such as urban monitoring, environmental assessment, and disaster response. Traditional CD mainly focuses on where change occurs, typically through pixel-level or object-level predictions. Over the past decade, progress has been driven by benchmark datasets such as [ 4 , 23 , 22 , 31 , 12 , 25 , 35 , 42 , 48 ] and architectural advances ranging from Siamese convolutional networks to transformer-based models such as ChangeFormer [ 2 ] and TSDT [ 22 ] . Despite these advances, obtaining reliable semantic supervision remains a major bottleneck, as fine-grained annotation of change types and descriptions is costly, slow, and difficult to scale. This limitation has motivated the emerging task of change captioning , which aims to explain what changed using natural language. Existing remote sensing change captioning datasets, however, typically describe the overall differences between two images at the scene level [ 22 , 5 ] . Such formulations are valuable, but they do not directly require fine-grained reasoning about a specific localized change. In many realistic settings, a model must determine whether a particular region changed, identify the semantic category of that change, and distinguish meaningful change from distractors or no-change regions. This requires region-grounded reasoning rather than only global scene summarization. To address this gap, we introduce RSRCC , a new benchmark for remote sensing change question answering built around localized, change-specific questions. Rather than asking for a single caption summarizing the entire image pair, RSRCC evaluates whether a model can reason about a particular semantic change instance. To the best of our knowledge, this is the first remote sensing change question answering benchmark designed explicitly for such fine-grained reasoning-based supervision. Figure 1 shows representative examples from the dataset. To construct RSRCC , we propose a hierarchical semi-supervised curation pipeline centered on Best-of- N ranking . Candidate change regions are first extracted from semantic segmentation masks using connected component analysis. They are then validated in two stages: fast semantic screening with a remote-sensing-tuned image-text encoder, followed by retrieval-augmented semantic verification with Best-of- N ranking for ambiguous cases. This design filters false and ambiguous changes, reducing manual annotation and improving consistency. Our main contributions are as follows: A new benchmark for localized semantic change. We introduce RSRCC , a high-quality remote sensing change question-answering benchmark built from LEVIR-CD [ 4 ] . Unlike prior datasets that emphasize global image-pair descriptions, RSRCC is constructed around localized, change-specific questions, with diverse semantic labels and visual patterns for training and evaluation of multimodal models. A hierarchical semi-supervised curation pipeline with Best-of- N ranking. We propose a scalable data construction framework that combines connected-component-based candidate extraction, image-text semantic screening, and retrieval-augmented Best-of- N ranking for ambiguity-aware validation. A theoretical interpretation of retrieval-guided validation. We provide an analysis showing how in-distribution retrieval and group-restricted conditioning can reduce reasoning bias by preserving local decision boundaries and limiting exposure to out-of-distribution examples (see Appendix A ). Table 1 compares RSRCC with existing remote sensing text-image datasets. While several prior datasets provide natural-language descriptions and some include bi-temporal imagery, they typically describe overall scene-level content or global before-after differences. In contrast, RSRCC is built around localized, change-specific supervision, where each example targets a particular semantic change instance. This makes RSRCC , to the best of our knowledge, the first benchmark in this space designed explicitly for fine-grained localized change. Table 1: Comparison with existing remote sensing text-image datasets. Dataset Year #Captions Details Temporal Localized Change UCM-Captions [ 26 ] 2016 10,500 ✗ ✗ ✗ RSICD [ 26 ] 2018 54,605 ✗ ✗ ✗ fMoW [ 7 ] 2018 N/A ✗ ✓ ✗ SpaceNet 7 [ 34 ] 2021 N/A ✗ ✓ ✗ S2Looking [ 31 ] 2021 N/A ✗ ✓ ✗ QFabric [ 36 ] 2021 N/A ✗ ✓ ✗ SpaceNet 8 [ 10 ] 2022 N/A ✗ ✓ ✗ LEVIR-CC [ 22 ] 2022 50,385 ✓ ✓ ✗ Dubai-CCD [ 21 ] 2022 2,500 ✓ ✓ ✗ RSICap [ 11 ] 2023 2,585 ✓ ✗ ✗ RSSM [ 44 ] 2024 5M ✓ ✗ ✗ VRSBench [ 20 ] 2024 29,614 ✓ ✗ ✗ WHU-CDC [ 38 ] 2024 37,170 ✓ ✓ ✗ XLRS-Bench [ 18 ] 2025 934 ✓ ✗ ✗ RSCC [ 5 ] 2025 62,351 ✓ ✓ ✗ RSRCC (Ours) 2026 126k ✓ ✓ ✓ 2 Related Work and Notations 2.1 Change Detection CD identifies differences between multi-temporal remote sensing images. Formally, given two co-registered images I_{t},I_{t+\Delta t}\in\mathbb{R}^{H\times W\times C} (1) acquired at times t and t+\Delta t , the goal is to estimate a change mask M=\mathcal{F}(I_{t},I_{t+\Delta t}), (2) where M\in\{0,1\}^{H\times W} and M_{ij}=1 indicates that pixel (i,j) has changed. Progress in CD has been strongly shaped by benchmark datasets. LEVIR-CD [ 4 ] remains one of the most widely used resources for high-resolution building change detection. LEVIR-MCI [ 23 ] extends this setting toward finer-grained semantic classes, while LEVIR-CC [ 22 ] introduces natural-language descriptions of visual changes. More recently, RSCC [

Read the complete text