Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation

Guanchun Wang, Chenxiao Wu, Xiangrong Zhang, Zelin Peng, Jianxun Lai, Tianyang Zhang, Xu Tang · 2026

Read Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation by Guanchun Wang, Chenxiao Wu, Xiangrong Zhang, Zelin Peng, Jianxun Lai, Tianyang Zhang, Xu Tang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26221.txt.

Introduction

Wang and six colleagues built SeeCo, a plug-in tool that sharpens open-vocabulary segmentation of satellite images without any extra training. It adjusts each model on the spot for the scene at hand, checking agreement across different views and across wording, so land-cover labels stay consistent. Tests on eight remote-sensing benchmarks showed steady gains over the unaided models.

Catalog record

CREATORGuanchun Wang, Chenxiao Wu, Xiangrong Zhang, Zelin Peng, Jianxun Lai, Tianyang Zhang, Xu Tang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34965
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.26221

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation Guanchun Wang, Chenxiao Wu, Xiangrong Zhang, Zelin Peng, Jianxun Lai, Tianyang Zhang, Xu Tang Abstract Open-vocabulary semantic segmentation (OVSS) in remote sensing images is a promising task that employs textual descriptions for identifying undefined land cover categories. Despite notable advances, existing methods typically employ a static inference paradigm, overlooking the distinct distribution of each scene, resulting in semantic ambiguity in diverse land covers and incomplete foreground activation. Motivated by this, we propose See king Co nsensus, termed SeeCo , a plug-and-play framework to boost the performance of training-free OVSS models in remote sensing images, which recalibrates arbitrary OVSS models on-the-fly by seeking dual consensus: geometric consensus learning (GCL) through multi-view consistent observations and semantic consensus learning (SCL) via textual description adaptive calibration, which assists collaborative recalibration of visual and textual semantics. The two consensus are injected via an online consensus injector (OCI), effectively alleviating the under-activation and semantic bias. SeeCo requires no specific training process, yet recalibrates semantic-geometric alignment for each unique scene during inference. Extensive experiments on eight remote sensing OVSS benchmarks show consistent gains, proving its effectiveness and universality. 1 Introduction Remote sensing semantic segmentation, a foundational task in Earth observation, plays an essential role in practical applications such as precision agriculture, hazard assessment, and urban planning 43 ; 24 ; 22 . With the progress of vision language models (VLMs), open-vocabulary semantic segmentation (OVSS) 37 ; 25 has attracted growing research awareness. By leveraging textual descriptions and pretrained VLMs, OVSS takes a detour from the traditional paradigms that rely on a closed set of pre-defined categories, providing a feasible solution for zero-shot land cover extraction from remote sensing images. Figure 1 : Comparison of open-vocabulary semantic segmentation results in remote sensing scenes from existing SOTA methods 18 , under different observed views (top) and different text descriptions (bottom). The arbitrary orientation and intra-class heterogeneity render it challenging to fully capture targets via only a single view or a category name. Existing OVSS methods can be broadly divided into fine-tuning and training-free methods. The former 8 ; 25 first fine-tunes image-level VLMs, especially for CLIP 27 , on closed-set semantic segmentation datasets with dense annotations to enable segmentation capability, and then infers the open-domain images. The latter aims to introduce the ability of dense prediction via a series of training-free strategies, such as modifying the architecture of CLIP 17 , or integrating CLIP with vision foundation models 18 ; 20 ; 21 , which are more appropriate for remote sensing images that require high costs of pixel-level annotations. Despite promising progress via representation resolution enhancement, the above methods follow a static inference paradigm that ignores the unique distribution of each RSI, thereby leading to the following challenges in complex scenes: Unlike natural scenarios, which are typically object-centric and have relatively fixed spatial arrangements, RSIs are captured from a bird’s eye view, resulting in land covers with arbitrary orientations and complex layouts, such as roads, buildings, and vehicles. As illustrated in Figure 1 , the static inference paradigm, constrained to a single observation perspective, fails to accommodate geometric variations, leading to incomplete activation in object regions. Therefore, we argue the following design principle: Guideline 1: A remote sensing OVSS framework should maintain rotation-invariant representation capabilities to match observations from bird’s-eye views. Land covers exhibit substantial intra-class heterogeneity. For instance, the category “building” encompasses both densely packed residential structures and sparse skyscrapers, resulting in a remarkable difference in visual appearance. Static textual descriptions further harm cross-modal matching in vision-language models, which are mainly pretrained on natural scenes while dealing with remote sensing captions, leading to semantic bias, as shown in Figure 1 . Previous studies 25 state that maintaining the text encoder of CLIP frozen can enhance generalization on open-vocabulary tasks, thereby retaining its robust textual embeddings across numerous categories. Hence, we formulate another design principle as follows: Guideline 2: A remote sensing OVSS framework should alleviate its semantic bias while keeping the text encoder frozen. Motivated by the above analysis, we propose Seeking Consensus , termed SeeCo , a plug-and-play and inference-time enhancement framework that can ameliorate most existing training-free OVSS methods. By simultaneously aligning semantic and geometric consensus during inference, SeeCo takes a detour from the limitations of static inference paradigms and realizes the dynamic adaptation of visual features and textual semantics, thereby improving remote sensing OVSS performance. More specifically, inspired by the multi-view observation of remote sensing images, we first introduce a geometric consensus learning (GCL) module that enforces rotation-variant self-supervised regularization, facilitating the model to produce consistent object representations across multi-orientation views, which significantly enhances the geometric robustness of object region features and alleviates the under-activation issue. Next, considering that a single category text cannot adequately capture land covers with intra-class diversity, we propose a semantic consensus learning (SCL) mechanism that mines rich knowledge through a multi-modal collaborative prompting strategy and dynamically recalibrates textual descriptions based on the distribution of each scene, thereby reducing semantic bias without tuning the frozen text encoder. Finally, we design a lightweight online consensus injector (OCI) that integrates the above consensus knowledge into the model during inference, enabling on-the-fly adaptive enhancement. The main contributions of this work can be summarized as follows: • We propose SeeCo, a plug-and-play framework for remote sensing OVSS through seeking geometric and semantic consensus on-the-fly during inference, which leads to substantial improvements in segmentation results by leveraging scene-specific feature distributions. • We develop a geometric consensus learning mechanism (GCL) and a semantic consensus learning mechanism (SCL) via rotation consistency-based regularization and textual description adaptive calibration, which assist collaborative recalibration of visual and textual semantics. The two consensus are injected online via a lightweight online consensus injector (OCI), effectively mitigating the issues of under-activation and semantic bias. • Extensive experiments on eight public remote sensing semantic segmentation benchmarks demonstrate that SeeCo, without any offline training or manual annotation, significantly outperforms state-of-the-art methods. 2 Related Work 2.1 Open-Vocabulary Semantic Segmentation Existing methods on open-vocabulary semantic segmentation are broadly categorized into fine-tuning and training-free paradigms. Fine-tuning OVSS Methods. The fine-tuning OVSS paradigm follows the manners of mask proposal classification 13 ; 9 or dense feature adaptation 19 ; 34 ; 8 to transfer the cross-modal representation in vision-language models (e.g., CLIP 27 ) to segmentation tasks. The former 13 ; 9 first organizes pixels into class-agnostic groups and then aligns regional vision features with text embeddings for OVSS. The latter 19 ; 34 ; 8 aims to directly perform pixel-level alignment between visual and textual features from vision-language models and enhances the semantic consistency by introducing multi-scale feature extraction or global context correspondence maps. Considering the properties of RSIs, previous studies improve OVSS through feature enhancement 35 , domain-specific pre-training 42 , and foundation model integration 10 ; 15 . Although such paradigms exhibit remarkable results, they rely on numerous pixel-level annotations and high computational costs, restricting rapid deployment in remote sensing applications. Training-free OVSS Methods. Training-free OVSS paradigms have gained increasing attention, aiming to directly exploit the inherent discriminative representation capacity of pre-trained VLMs without additional training, which fall into two technical streams. One stream focuses on refining the intrinsic feature representations of CLIP via spatial coherence

Read the complete text