SOPSeg: Prompt-based Small Object Instance Segmentation in Remote Sensing Imagery
Chenhao Wang, Yingrui Ji, Yu Meng, Yunjian Zhang, Yao Zhu · 2025
Read SOPSeg: Prompt-based Small Object Instance Segmentation in Remote Sensing Imagery by Chenhao Wang, Yingrui Ji, Yu Meng, Yunjian Zhang, Yao Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-03002.txt.
Introduction
Chenhao Wang and 4 co-authors introduced a 2025 method called SOPSeg for outlining tiny objects in images taken from above. Satellite and drone pictures guide city planning, environmental checks, and disaster response. Their hardest targets are small objects, where drawing an exact outline around each one is slow and costly to label by hand.
General tools like the Segment Anything Model lose fine detail on such targets because their internal picture is coarse. SOPSeg magnifies promising regions, predicts edges, and refines outlines step by step. It also accepts the rotated box prompts common in remote sensing work. The authors built a new dataset for this task from an existing aerial benchmark. Their method beat rivals and should speed future dataset building.
Catalog record
| CREATOR | Chenhao Wang, Yingrui Ji, Yu Meng, Yunjian Zhang, Yao Zhu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-03002
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SOPSeg: Prompt-based Small Object Instance Segmentation in Remote Sensing Imagery Chenhao Wang, Yingrui Ji, Yu Meng, Yunjian Zhang, Yao Zhu Abstract Extracting small objects from remote sensing imagery plays a vital role in various applications, including urban planning, environmental monitoring, and disaster management. While current research primarily focuses on small object detection, instance segmentation for small objects remains underexplored, with no dedicated datasets available. This gap stems from the technical challenges and high costs of pixel-level annotation for small objects. While the Segment Anything Model (SAM) demonstrates impressive zero-shot generalization, its performance on small-object segmentation deteriorates significantly, largely due to the coarse 1/16 feature resolution that causes severe loss of fine spatial details. To this end, we propose SOPSeg, a prompt-based framework specifically designed for small object segmentation in remote sensing imagery. It incorporates a region-adaptive magnification strategy to preserve fine-grained details, and employs a customized decoder that integrates edge prediction and progressive refinement for accurate boundary delineation. Moreover, we introduce a novel prompting mechanism tailored to the oriented bounding boxes widely adopted in remote sensing applications. SOPSeg outperforms existing methods in small object segmentation and facilitates efficient dataset construction for remote sensing tasks. We further construct a comprehensive small object instance segmentation dataset based on SODA-A, and will release both the model and dataset to support future research. Introduction Remote sensing imagery plays a critical role in a wide range of real-world applications, including urban planning, environmental monitoring, and precision agriculture. Among the targets of interest in these applications, small objects such as vehicles, plane, and ships typically occupy no more than 32×32 pixels in high-resolution imagery, yet they convey essential semantic and operational information for downstream tasks. Consequently, accurately extracting small objects is of great importance, but remains a highly challenging task due to their limited size and complex visual characteristics. Benchmarks such as SODA-A ( Cheng et al. 2023 ) have significantly advanced small object detection in remote sensing imagery. However, they provide only bounding box annotations, which constrain models to coarse localization and fail to capture precise object shapes. Consequently, most existing works focus on object detection rather than instance segmentation, limiting fine-grained scene understanding. Instance segmentation for small objects remains largely underexplored, primarily due to the lack of suitable datasets. Constructing such datasets is highly labor-intensive, error-prone, and requires substantial domain expertise. Although the Segment Anything Model (SAM) ( Kirillov et al. 2023 ) , trained on over one billion masks, demonstrates strong zero-shot generalization capabilities, its direct application to high-resolution remote sensing imagery leads to notable performance degradation for small objects. We attribute this limitation to the architectural design of SAM: its vision transformer encoder downsamples input images to 1/16 of the original resolution to reduce computational cost. While effective for typical object sizes, this aggressive downsampling results in the loss of fine-grained details that are critical for accurately identifying small targets. To this end, we propose SOPSeg ( S mall O bject P rompted Seg mentation), a novel framework that adapts SAM for robust small-object instance segmentation in remote sensing imagery. Our approach introduces three key innovations: (1) Region-adaptive magnification, which adaptively crops and resizes object regions to preserve fine details lost in downsampling, enabling accurate segmentation of small instances with minimal overhead; (2) An edge-aware decoder, which integrates boundary prediction and progressive multi-scale refinement to produce sharper and more accurate object masks; (3) An oriented prompting mechanism, which enables the use of rotated bounding boxes common in aerial imagery, improving SAM’s ability to handle objects at arbitrary orientations. We train and validate SOPSeg on the iSAID dataset ( Waqas Zamir et al. 2019 ) , selecting 7 out of 15 categories that best represent small object challenges in remote sensing imagery. Generalization ability is further evaluated on the NWPU-VHR10 ( Su et al. 2019 ) and SAT-MTB ( Li et al. 2023b ) benchmarks. Experimental results show that SOPSeg significantly outperforms the original SAM and other prompt-based segmentation methods across all datasets. Figure 1: Overview of the SOPSeg Framework. The input remote sensing image is first cropped and resized into multiple high-resolution patches of uniform size. These patches are then fed into the SAM image encoder, where Position Embedding Interpolation (PEI) is applied to support arbitrary input sizes. An oriented prompt, consisting of a horizontal bounding box and three keypoints aligned with the orientation of the object, is encoded via the SAM prompt encoder to guide segmentation. Coarse masks and edges are generated and refined progressively to yield accurate high-resolution segmentation. \mathbf{F_{d}} and \mathbf{F_{s}} denote deep image features from decoder and shallow image features from image encoder. To demonstrate its practical utility, we further apply SOPSeg to assist in constructing a small object instance segmentation dataset. Specifically, we automatically generate approximately 709k instance masks for small objects based on images and oriented bounding boxes from the SODA-A dataset, followed by manual filtering to remove a small number of abnormal annotations. The resulting dataset ReSOS ( Re mote Sensing S mall O bject S egmentation), represents the first large-scale instance segmentation benchmark focused on small objects in remote sensing imagery. We plan to publicly release both the model and the dataset to provide training and evaluation resources for future research on small object analysis. In summary, our contributions are threefold: • We propose SOPSeg, a prompt-based framework that adapts SAM for small object instance segmentation, integrating region-adaptive magnification and edge-aware refinement decoding to enhance mask accuracy. • We develop an oriented prompting mechanism enabling accurate segmentation of objects at arbitrary orientations. • We construct and release ReSOS dataset, the first large-scale instance segmentation dataset specifically designed for small objects in remote sensing. It contains pixel-level annotations for over 709k instances and aims to support future research. Related Work Small Object Detection and Segmentation in Remote Sensing. Small object analysis in remote sensing has attracted significant research interest due to its practical importance. Early detection methods relied on hand-crafted features and traditional machine learning classifiers ( Cheng and Han 2016 ) . However, these approaches struggled with the complex backgrounds and varying scales characteristic of aerial imagery. Ding ( Ding et al. 2019 ) proposed a rotation-invariant detector specifically designed for aerial images, while Yang ( Yang et al. 2019 ) introduced SCRDet to handle the multi-scale and multi-orientation challenges. Recent methods have focused on feature enhancement strategies. RMSIN ( Liu et al. 2024 ) employs interaction modules to effectively capture complex spatial scales and orientations for accurate segmentation in remote sensing imagery. For instance, FCOS-RS ( Li et al. 2020 ) adapts the anchor-free FCOS detector for remote sensing by incorporating multi-scale feature fusion. Similarly, Oriented R-CNN ( Xie et al. 2021 ) extends Faster R-CNN ( Ren et al. 2016 ) with oriented region proposals to better capture arbitrarily oriented objects. Despite progress in detection, instance segmentation of small objects remains largely unexplored. The few existing works primarily focus on specific object categories. Zhang ( Zhang et al. 2017 ) developed a ship instance segmentation method using polar coordinates, while Zhao ( Zhao, Persello, and Stein 2021 ) proposed building extraction techniques. UGBS ( Yang et al. 2024 ) explored interactive user guidance mechanisms to achieve more accurate building segmentation from high-resolution