DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning

Yelin Wang 1,* , Zijia Song 2,* , Chuanguang Yang 3,† , Miaoyu Wang 4 , Zhulin An 3,† , Libo Huang 3 , and Yongjun Xu 3 · 2026

Read DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning by Yelin Wang 1,* , Zijia Song 2,* , Chuanguang Yang 3,† , Miaoyu Wang 4 , Zhulin An 3,† , Libo Huang 3 , and Yongjun Xu 3 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27410.txt.

Introduction

Wang leads a team making satellite image change captioning focus on the differences that matter. The task asks a model to describe, in words, what changed between two remote sensing images of the same place taken at different times. Standard training follows plain autoregressive generation, which rewards easy vocabulary over discriminative change features. Their Difference Feature Modeling framework retrains the objective around contrast. In 2026, a text-guided gated contrastive loss steers the vision encoder toward features the caption actually needs.

A pre-trained change detection model donates stable knowledge about where change occurs. A joint feature module fuses difference representations across scales. Experiments on multiple datasets show the approach works, with gains over prior captioning models. Captions grow more precise about what appeared, vanished, or moved. Earth observation gains a narrator that watches the change, not the scenery.

Catalog record

CREATORYelin Wang 1,* , Zijia Song 2,* , Chuanguang Yang 3,† , Miaoyu Wang 4 , Zhulin An 3,† , Libo Huang 3 , and Yongjun Xu 3
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.27410

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning Yelin Wang 1,* , Zijia Song 2,* , Chuanguang Yang 3,† , Miaoyu Wang 4 , Zhulin An 3,† , Libo Huang 3 , and Yongjun Xu 3 Abstract The primary goal of Remote Sensing Image Change Captioning (RSICC) is to automatically generate descriptions of changes between remote sensing images captured at different time points. Existing models still rely on a single autoregressive generation paradigm, which tends to prioritize learning easily generated vocabulary over capturing discriminative differences between images. To address this, we reframe the training paradigm and propose a novel Difference Feature Modeling (DFM) framework. Specifically, we introduce a Text-guided Gated Contrastive Loss (TGCL) to guide the vision encoder to extract critical features from a text-modal perspective. Additionally, we incorporate a pre-trained Change Detection model to transfer stable change detection knowledge. In order to further enhance the representation, we design a Joint Feature Modeling (JFM) module to achieve the fusion of multi-scale difference representations, thereby capturing comprehensive spatiotemporal variations between multi-temporal images. Extensive experiments on multiple datasets demonstrate the effectiveness of our approach. I Introduction Remote Sensing Image Change Captioning (RSICC) is a critical task in the field of remote sensing . It involves analyzing multi-temporal remote sensing images of the same area and describing surface changes through natural language. Compared to Remote Sensing Image Change Detection (RSICD), which only requires segmenting the changed regions between multi-temporal images, RSICC leverages natural language to provide a more detailed characterization of changed object categories, positional relationships between objects, and dynamics of changing objects (e.g., additions or disappearances). RSICC significantly enhances the interpretability of change-related information and holds substantial application potential in various domains, including land planning, surface dynamics monitoring, and urban expansion research. Fig. 1: Challenges of previous models for RSICC and brief display of our method for generating semantically precise captions. Compared to conventional image captioning, RSICC is a more challenging multi-modal task. It requires not only accurate interpretation of image content but also the ability to capture changes between multi-temporal images and generate corresponding natural language descriptions. Current mainstream methods [ 1 , 2 , 3 , 4 ] typically adopt an encoder-decoder architecture, where visual features are encoded and decoded into textual descriptions. However, the critical challenge lies in guiding the model to effectively represent the visual changes between images while ensuring that these representations are sufficiently informative for the text decoder to generate semantically precise captions [ 5 , 6 , 7 , 8 , 9 ] . To better characterize image changes, prior work explores change-aware captioning [ 10 , 11 ] . PromptCC [ 2 ] decouples detection and description and adopts a multi-prompt strategy [ 12 , 13 , 14 ] . RSCaMa [ 1 ] applies State Space Models (SSM) for iterative spatial perception and temporal interaction. However, as shown in Figure 4 , these methods still rely on supervised autoregressive generation: captions are produced by a text decoder and optimized with a single loss across the vision encoder, change-focused neck, and decoder, which can induce model laziness. Moreover, the autoregressive loss uniformly supervises all tokens, ignoring position-dependent generation difficulty [ 15 , 16 ] . Fig. 2: Flowchart of Difference Feature Modeling (DFM) framework for RSICC. A novel Text-Guided Gated Contrastive Loss is proposed, and the change detection knowledge is transferred through new designed Joint Feature Modeling module. To address these challenges, we propose Difference Feature Modeling (DFM). Beyond the generation paradigm, DFM introduces cross-modal contrastive learning and integrates change detection knowledge to emphasize visually significant changes. Specifically, we add a text encoder and design a Text-guided Gated Contrastive Loss (TGCL) to align change-descriptive captions with difference features from multi-temporal images. A gating mechanism filters out “no-change” captions, preventing them from misleading optimization. In addition, we transfer spatial knowledge from pre-trained Change Detection models to strengthen change-related representations, and design a Joint Feature Modeling module (JFM) to fuse multi-scale features. By directly applying TGCL to the vision encoder, DFM enforces stronger constraints and drives the model toward visually grounded changes. The main contributions of this work are summarized as follows: • We identify model laziness in existing RSICC methods , where autoregressive generation tends to over-rely on generic linguistic patterns instead of visual evidence. To mitigate this issue, we propose a Text-guided Gated Contrastive Loss and leverage pre-trained Change Detection models to enforce visually grounded change understanding. • We introduce a Joint Feature Modeling module that jointly captures pixel-level and structural difference features across multiple scales, enabling more accurate and context-aware change captioning. • Extensive experiments on multiple benchmarks demonstrate the effectiveness of the proposed Difference Feature Modeling framework, achieving consistent improvements over state-of-the-art methods, particularly in terms of CIDEr-D . II Related Works II-A Remote Sensing Change Detection Remote sensing change detection aims to identify changes between multi-temporal images and generate pixel-level change masks [ 17 , 18 , 19 ] . Early approaches mainly relied on pointwise classification and handcrafted features [ 20 , 21 , 22 , 23 ] . With the advent of deep learning, CNN-based methods substantially improved change detection performance due to their strong representation and nonlinear modeling capabilities [ 5 , 6 , 7 ] . More recently, attention mechanisms and transformer architectures have been introduced to capture spatial–temporal dependencies, achieving further performance gains [ 24 , 25 , 26 , 27 ] . In parallel, emerging paradigms such as diffusion models and Mamba architectures have driven rapid progress in change detection research [ 28 , 29 , 30 ] . Closely related to change detection, change captioning focuses on describing changes at the semantic level rather than pixel-wise localization [ 2 ] . II-B Remote Sensing Image Change Captioning Remote Sensing Image Change Captioning is a relatively new research direction in remote sensing. Chouaf et al . [ 10 ] first explored RSICC on a private dataset using a CNN–RNN encoder–decoder framework. Subsequent works extended this paradigm by introducing different fusion strategies and architectures, including feature-level fusion and transformer-based models [ 11 , 31 ] . Notably, Liu et al . [ 31 ] released the large-scale LEVIR-CC dataset and established benchmark methods based on dual-branch transformer encoders. Further improvements were achieved through multi-scale and difference-aware modeling to enhance the perception of changed objects [ 15 ] . More recently, Mamba-based architectures have been explored for dual-temporal spatiotemporal feature extraction [ 16 ] . Despite promising progress, existing RSICC methods still struggle with generating accurate, semantically rich, and detail-consistent change descriptions. Motivated by these limitations, we propose a series of improvements to address the insufficient and inaccurate semantic modeling in prior work. III Methodology III-A Overview of Framework Previous methods predominantly adopt autoregressive generation, often leading to model laziness and repetitive captions rather than precise change descriptions. To address this, we propose Difference Feature Modeling , which guides the model to focus on inter-image changes (Fig. 2 ). DFM introduces a Text-guided Gated Contrastive Loss to emphasize text-relevant change content and a Joint Feature Modeling module to refine change-descriptive representations using change detection priors. The end-to-end training procedure is summarized in Alg. 1 . III-B Change Detection Branch We

Read the complete text