OpenRSD: Towards Open-prompts for Object Detection in Remote Sensing Images
Ziyue Huang, Yongchao Feng, Shuai Yang, Ziqi Liu, Qingjie Liu, Yunhong Wang · 2025
Read OpenRSD: Towards Open-prompts for Object Detection in Remote Sensing Images by Ziyue Huang, Yongchao Feng, Shuai Yang, Ziqi Liu, Qingjie Liu, Yunhong Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06146.txt.
Introduction
Ziyue Huang, Yongchao Feng, Shuai Yang, and co-authors present a detection system for satellite and aerial images that answers to open text prompts in this 2025 paper. Standard detectors only find the object classes they were trained on, which limits their use across new regions and tasks. Open-vocabulary methods tie images to text so users can ask for new classes in words.
Their framework, OpenRSD, accepts text and other prompt types and runs detection heads for both level and tilted boxes, since objects in overhead views sit at any angle. A staged training plan builds broad skill before fine skill. On seven public datasets the system beat the leading comparison tool by 8.7 percent in average precision. It also ran at about twenty-one frames per second, fast enough for large-area scanning. The authors release the work as a base for real-time remote sensing analysis.
Catalog record
| CREATOR | Ziyue Huang, Yongchao Feng, Shuai Yang, Ziqi Liu, Qingjie Liu, Yunhong Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-06146
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OpenRSD: Towards Open-prompts for Object Detection in Remote Sensing Images Ziyue Huang, Yongchao Feng, Shuai Yang, Ziqi Liu, Qingjie Liu, Yunhong Wang Abstract Remote sensing object detection has made significant progress, but most studies still focus on closed-set detection, limiting generalization across diverse datasets. Open-vocabulary object detection (OVD) provides a solution by leveraging multimodal associations between text prompts and visual features. However, existing OVD methods for remote sensing (RS) images are constrained by small-scale datasets and fail to address the unique challenges of remote sensing interpretation, include oriented object detection and the need for both high precision and real-time performance in diverse scenarios. To tackle these challenges, we propose OpenRSD, a universal open-prompt RS object detection framework. OpenRSD supports multimodal prompts and integrates multi-task detection heads to balance accuracy and real-time requirements. Additionally, we design a multi-stage training pipeline to enhance the generalization of model. Evaluated on seven public datasets, OpenRSD demonstrates superior performance in oriented and horizontal bounding box detection, with real-time inference capabilities suitable for large-scale RS image analysis. Compared to YOLO-World, OpenRSD exhibits an 8.7% higher average precision and achieves an inference speed of 20.8 FPS. Codes and models will be released. 1 Introduction As an important task of remote sensing (RS) image interpretation, remote sensing object detection has witnessed significant advancements in recent years [ 44 , 29 ] . Nonetheless, the majority of existing studies have concentrated on closed-set object detection, which aims to optimize performance on a fixed set of categories within specific datasets by refining model architectures [ 44 ] or loss functions [ 46 ] . These models frequently exhibit poor generalization capabilities across diverse datasets and are limited to detecting objects belonging to predefined categories. Open-vocabulary object detection (OVD) [ 24 , 7 ] in natural scenes present a viable solution to this limitation by establishing multimodal associations between text prompts and visual features, thereby relaxing the limitations of predefined categories. Furthermore, they leverage large-scale datasets such as image-text pairs [ 30 ] , high-quality detection datasets [ 20 ] , and grounding data [ 24 ] for training, enabling robust generalization across various scences. Figure 1 : We compare the precision and inference speed of OpenRSD against other methods, with inference speed tested on a single 2080Ti GPU. Recent studies have explored OVD models for RS images [ 42 , 32 , 22 ] ; however, these efforts are constrained by the use of small-scale datasets, which are inadequate to support robust generalization. Furthermore, they fail to address the unique requirements of RS scenarios, such as detecting small objects and precisely object orientations. This gap motivate us to develop a universal RS object detector with functional flexibility and strong cross-domain detection capabilities, serving as a foundational model for RS image interpretation. To this end, we propose a multimodal prompt-based object detector and introduce a multi-stage training pipeline to strengthen its generalization. We improve upon the RTMDet [ 29 ] framework and propose the OpenRSD , which achieves the best performance in both overall accuracy and inference speed, as shown in Fig. 1 . A prompt construction module is developed, utilizing pretrained models include SkyCLIP [ 41 ] and DINOv2 [ 31 ] to extract text and image prompt embeddings. The image prompts aim to reduce reliance on expert knowledge, which is particularly important since objects and fine-grained categories in RS images are often rare in everyday contexts. By incorporating random sampling during training, the detector is enabled to identify relevant object based on arbitrary prompts. We design two detection heads with multi-task: alignment head and fusion head. The alignment head performs classification by computing the similarity between detection features and prompt embeddings, offering faster inference and easy scalability for expanding the number of categories. The fusion head introduces cross-modal interactions, achieving better detection performance. Meanwhile, we integrate both horizontal and oriented bounding box regression tasks into one framework. Both heads are trained simultaneously with minimal impact on training time, offering a flexible design to meet diverse application requirements in RS scenarios. To fully leverage existing datasets and enable the detector to robust detect objects in general scenarios, we propose a multi-stage training pipeline. Existing RS datasets are fragmentation and diversity, encompassing numerous specialized datasets, such as HRSC2016 [ 25 ] for ship and SpaceNet [ 37 ] for building. This heterogeneity poses substantial challenges for jointly trained detectors, as models trained on individual datasets often fail to generalize effectively to other scenarios, leading to severe false positives and false negatives. To address these challenges, we first pretrain the detector using both labeled and unlabeled data to adapt to RS data and the detection task. Subsequently, we finetune detector and employ it to generate pseudo-labels, and leverage SkyCLIP [ 41 ] for filtering. Finally, we continue fine-tuning detector using the complete self-annotated dataset integrated with labeled data, endowing it with strong generalization performance. Our contributions are summarized as follows: (1) We propose an Open -Prompt R emote S ensing object D etector (OpenRSD) that supports both oriented and horizontal object detection using either image or text prompts. To balance inference speed and high-precision detection, we introduce an alignment head and a fusion head, each specifically designed to address these distinct requirements. (2) We design a multi-stage training pipeline consists of pretraining, finetuning, and self-training to enhance the detector’s generalization and cross-domain detection capabilities. Furthermore, we construct a large-scale training dataset, ORSD+, which comprises over 470k images spanning 200 categories. (3) We evaluate OpenRSD on seven public datasets. For oriented object detection, OpenRSD demonstrates superior average performance. For horizontal bounding box detection, OpenRSD achieves performance comparable to the high-precision baseline model, Grounding DINO [ 24 ] , but with significantly higher inference speed. It runs at a inference speed of 20.8 FPS, making it well-suited for large-scale RS image analysis. Figure 2 : OpenRSD consists of three components: multi-scale image feature encoding, prompt construction, and multi-task detection heads. The prompt construction randomly samples multiple text or image prompts for each category and encodes them into prompt embeddings. The multi-task detection heads leverage the correlation between prompt embeddings and image features to perform detection, seamlessly integrating fusion-based and alignment-based open-prompt structures within one framework. This design enables support for a wide range of prompt-based classification and regression tasks. The alignment head offers higher speed and greater vocabulary scalability, while the fusion head achieves better precision, allowing the model to adapt to different application scenarios. To enhance generalization, we further employ a multi-stage training pipeline. 2 Related Work 2.1 Open-Set Object Detection Open-set object detection models utilize multimodal prompts [ 24 , 16 ] to detect relevant objects. Early works [ 12 ] adhered to strict OVD settings [ 49 ] , training in base classes and testing in unseen classes, using conventional datasets like COCO [ 23 ] for research. Recent studies have attempted to enhance generality by expanding available datasets through multitask learning [ 24 , 47 ] or self-training [ 30 ] , and by conducting large-scale image-text alignment pre-training. In terms of model architecture, GLIP [ 20 ] , Grounding DINO [ 24 ] , and OV-DINO [ 39 ] introduced deep cross-modal fusion, while YOLO-World incorporated lightweight feature interaction