DINO Soars: DINOv3 for Open-Vocabulary Semantic Segmentation of Remote Sensing Imagery
Ryan Faulkenberry, Saurabh Prasad · 2026
Read DINO Soars: DINOv3 for Open-Vocabulary Semantic Segmentation of Remote Sensing Imagery by Ryan Faulkenberry, Saurabh Prasad (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03175.txt.
Introduction
Ryan Faulkenberry and Saurabh Prasad build an open-vocabulary segmentation model for remote sensing images that needs no fine-tuning on remote sensing data. Labeled satellite and aerial data are scarce and costly, yet land-cover maps need dense labels.
Their CAFe-DINO model uses the DINOv3 backbone with text-image scoring, cost aggregation, and training-free upsampling, tuned only on a COCO-Stuff subset aimed at the domain. It reached state-of-the-art results on key remote sensing benchmarks, beating methods that were fine-tuned on remote sensing data itself. Code and data are public, making the approach easy to test and reuse.
Catalog record
| CREATOR | Ryan Faulkenberry, Saurabh Prasad |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03175
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DINO Soars: DINOv3 for Open-Vocabulary Semantic Segmentation of Remote Sensing Imagery Ryan Faulkenberry, Saurabh Prasad Abstract The remote sensing (RS) domain suffers from a lack of densely labeled datasets, which are costly to obtain. Thus, models that can segment RS imagery well without supervised fine-tuning are valuable, but existing solutions fall behind supervised methods. Recently, DINOv3 30 surpassed SOTA RS foundation models on the GEO-bench segmentation benchmark without pre-training on RS data. Additionally, DINO.txt 16 has enabled open vocabulary semantic segmentation (OVSS) with the DINOv3 backbone. We leverage these developments to form an OVSS model for RS imagery, free of RS-domain fine-tuning. Our model, CAFe-DINO (Cost Aggregation + Feature Upsampling with DINO) exploits the strong OVSS performance of DINOv3 for RS imagery via cost aggregation and training-free upsampling of text-image similarity scores. The robust latent of the DINOv3 backbone eliminates the need for fine-tuning on RS imagery; we instead fine-tune our model on a RS-targeted subset of COCO-Stuff. CAFe-DINO achieves state-of-the-art performance on key RS segmentation datasets, outperforming OVSS methods fine-tuned on RS data. Our code and data are publicly available at https://github.com/rfaulk/DINO_Soars. Figure 1 : Open-vocabulary segmentation maps from our model, CAFe-DINO. CAFe-DINO can accurately segment remote sensing scenes with arbitrary semantic classes, despite zero training on remote sensing imagery. 1 Introduction (a) DINOv3.txt (b) DINOv3.txt + CA + AnyUp (Ours) (c) Ground Truth Figure 2 : DINOv3.txt alone is not a strong OVSS model for remote sensing Remote sensing (RS) imagery has revolutionized our ability to observe the planet. It has equipped us with the ability to monitor our environment, optimize agriculture, and respond to natural disasters. RS imagery does not enjoy the same wealth of densely labeled datasets as natural imagery, which has led to a body of work on adapting models trained on natural imagery to fit the unique characteristics of RS imagery, with little or no fine-tuning. For semantic segmentation tasks in RS, the latest and most successful of these models are Open Vocabulary Semantic Segmentation (OVSS) models. They leverage vision-language models (VLMs) that are pretrained to align a large corpus of image and text pairs. These models, imbued with a large “vocabulary” from their pretraining, can be prompted with the name of a desired class to create a similarity map between that class and an input image, enabling zero-shot segmentation of arbitrary classes. The recent release of DINOv3 30 included a text encoder aligned with DINOv3 image features, enabling OVSS via alignment of image features and text embeddings of semantic classes 16 . Their “DINOv3.txt” model surpasses OVSS state-of-the-art by a large margin on natural imagery benchmarks. DINOv3 has also shown remarkable adaptability to RS tasks without any pre-training on RS imagery, outperforming RS-specific foundation models (e.g. Prithvi 33 , DOFA 42 ), as well as a sister version of DINOv3 trained on RS imagery, on five of six GEO-Bench 18 segmentation tasks. It is noteworthy that DINOv3 achieved this while using only the RGB bands of these mostly multi-spectral datasets. This suggests that natural image training at scale can overcome RS-specific obstacles such as scale/resolution differences, sensor noise, and varying spectral characteristics. Motivated by these results, we look to DINOv3.txt as a starting point for a RS OVSS model trained without RS imagery. Despite its pedigree on natural imagery, DINOv3.txt struggles with OVSS on RS imagery, as we show in Fig. 2(a) and Table 2 . We unlock the RS capability of DINOv3 for OVSS by adding a cost aggregation module to semantically and spatially refine the similarity maps of DINOv3.txt. Additionally, motivated by the recent success of feature upsampling 13 ; 32 ; 9 ; 39 , we incorporate AnyUp 39 to upsample the aggregated features directly without fine-tuning. The combination of AnyUp, which is training-free, and the cost aggregation module, which learns only to denoise similarity maps from the existing DINO vision/text encoders, creates a synergistic architecture in which RS fine-tuning is not required. Contributions: 1. We introduce CAFe-DINO, a cost-aggregation and feature upsampling model for the DINOv3 backbone that does not require RS fine-tuning. We demonstrate the efficacy of cost aggregation and feature upsampling for OVSS of RS imagery. 2. Leveraging the capability of DINOv3 on RS imagery, we train CAFe-DINO on only natural imagery with an RS-targeted subset of the COCO-Stuff 3 dataset. 3. We show state of the art OVSS performance on a set of multiclass RS segmentation tasks with CAFe-DINO. (a) An overview of our method. Class-wise similarity maps from DINOv3 are aggregated and upsampled to form a segmentation prediction. (b) An overview of the cost aggregation network. Cost maps are projected and propagated through a series of aggregation blocks. Figure 3 : Details of the CAFe-DINO architecture. 2 Related Work 2.1 Open Vocabulary Semantic Segmentation Open vocabulary semantic segmentation (OVSS) 47 has advanced significantly with vision-language models (VLMs) 28 ; 21 ; 6 ; 22 that imbue an image encoder with text-conditioned reasoning. CLIP 28 introduced a contrastive method for aligning image and text embeddings, attaining state-of-the-art performance on zero-shot image classification in its time. CLIP has since been adapted for application to segmentation tasks and forms the backbone of most OVSS models to this day 14 ; 10 ; 20 ; 25 ; 43 . Early adaptations such as SegCLIP 46 and PACL 27 introduced image patches rather than a global image embedding as an alignment objective. Later work added more sophisticated techniques, such as mask proposal-based region segmentation 24 ; 2 ; 17 ; 29 ; 31 , and CAT-Seg 8 , which incorporated cost aggregation (See Section 2.5 ) of aligned CLIP features for segmentation. Some methods have managed to eschew CLIP and still achieve good performance. One such class of methods 36 ; 19 apply text-guided diffusion models to generate embeddings for arbitrary classes. Despite these advances, most OVSS methods still struggle with fine-grained localization and domain shift, particularly when deployed outside natural image distributions such as RS. 2.2 OVSS for Remote Sensing Because OVSS for remote sensing has only recently emerged, there are only a few methods published. GSNet 44 uses a DINO-based RS backbone as guidance for a frozen CLIP model and trains on a newly introduced large-scale semantic segmentation dataset for RS. AerOSeg 12 uses SAM to spatially refine CLIP features and guide upsampling. T2ASeg 38 uses gradient activation maps to augment CLIP embeddings. OVRS 4 applies some RS-targeted modifications to the CAT-Seg framework, such as rotational invariance encoding and feature-guided upsampling. SegEarth-OV 23 directly upsamples CLIP features with guidance from the input image, then subtracts the [CLS] token to extract localized features; notably, they require only self-supervised 35 training on RS data, though on a per-dataset basis. Collectively, these methods demonstrate the promise of OVSS in RS, but are limited by the need to train with RS imagery. 2.3 DINOv3 DINOv3 30 is a very large vision foundation model applicable to a variety of visual tasks. DINOv3 is based on a ViT 11 scaled to 7B parameters, trained via self-supervised learning (SSL) on a pool of over 1 billion unlabeled images. The 7B-parameter model is distilled into versions matching the usual ViT sizes (ViT-H, ViT-B, etc). DINOv3 outperforms established geospatial foundation models (GFMs) 41 ; 34 ; 33