Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection

Yu Guo, Shengfeng He, Yuxu Lu, Haonan An, Yihang Tao, Huilin Zhu, Jingxian Liu, Yuguang Fang, City University of Hong Kong Singapore Management University · 2025

Read Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection by Yu Guo, Shengfeng He, Yuxu Lu, Haonan An, Yihang Tao, Huilin Zhu, Jingxian Liu, Yuguang Fang, City University of Hong Kong Singapore Management University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-20745.txt.

Introduction

Ships and other objects at sea are hard to spot in images, and labeled training photos are scarce. Yu Guo and seven co-authors address both problems with Neptune-X, a 2025 framework for maritime object detection. The approach generates synthetic training scenes and then selects the most useful ones.

A generator called X-to-Maritime creates varied, realistic sea scenes from several kinds of input. A special attention module models the boundary where objects meet water, a common source of detection errors. A task-aware selection step then picks synthetic samples that help training most. The authors report better detection and stronger generalization across object types, viewpoints, locations, and imaging conditions than training on limited real data alone.

Catalog record

CREATORYu Guo, Shengfeng He, Yuxu Lu, Haonan An, Yihang Tao, Huilin Zhu, Jingxian Liu, Yuguang Fang, City University of Hong Kong Singapore Management University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-20745

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection Yu Guo, Shengfeng He, Yuxu Lu, Haonan An, Yihang Tao, Huilin Zhu, Jingxian Liu, Yuguang Fang, City University of Hong Kong Singapore Management University Abstract Maritime object detection is essential for navigation safety, surveillance, and autonomous operations, yet constrained by two key challenges: the scarcity of annotated maritime data and poor generalization across various maritime attributes (e.g., object category, viewpoint, location, and imaging environment). To address these challenges, we propose Neptune-X, a data-centric generative-selection framework that enhances training effectiveness by leveraging synthetic data generation with task-aware sample selection. From the generation perspective, we develop X-to-Maritime, a multi-modality-conditioned generative model that synthesizes diverse and realistic maritime scenes. A key component is the Bidirectional Object-Water Attention module, which captures boundary interactions between objects and their aquatic surroundings to improve visual fidelity. To further improve downstream tasking performance, we propose Attribute-correlated Active Sampling, which dynamically selects synthetic samples based on their task relevance. To support robust benchmarking, we construct the Maritime Generation Dataset, the first dataset tailored for generative maritime learning, encompassing a wide range of semantic conditions. Extensive experiments demonstrate that our approach sets a new benchmark in maritime scene synthesis, significantly improving detection accuracy, particularly in challenging and previously underrepresented settings. 1 Introduction Object detection is a fundamental technology for maritime environmental perception, enabling the identification of object categories and the localization of bounding boxes in images captured by imaging systems deployed on various facilities, such as surface vessels, coastal infrastructure, and aerial platforms. It plays a key role in a variety of maritime applications, including autonomous or assisted navigation for surface ships [ 43 ] , intelligent video surveillance for coastal facilities [ 40 ] , and autonomous inspection using Unmanned Aerial Vehicles (UAVs) [ 44 ] . Despite rapid advances in deep learning-based object detection [ 39 , 37 , 23 ] , the generalization capability of these models remains heavily dependent on the scale and diversity of annotated training data. Specifically, maritime-specific object detection datasets face two core limitations. First , the acquisition and annotation process is costly and labor-intensive. Different from land scenarios, data collection from heterogeneous platforms such as ships, UAVs, and stationary coastal cameras requires significant operational resources. In addition, the manual annotation of bounding boxes limits dataset scalability. Second , existing datasets exhibit large disparities in training difficulty across multiple attributes, including imaging conditions, viewpoints, water environments, and object categories. These imbalances are driven by uneven sample distributions and systematic biases in data collection, resulting in poor generalization to rare or complex maritime scenarios. Figure 1: We introduce Neptune-X, a generation-selection framework for robust maritime object detection. (a) It enables the first multi-modality-conditioned data generation, supporting diverse and controllable maritime scene synthesis. (b) Our active selection strategy leverages the Attribute-correlated Training Difficulty Factor (ATDF), which correlates with detection performance and guides the selection of high-value synthetic samples to optimize downstream detector accuracy. To mitigate data limitations, traditional data augmentation techniques [ 51 , 48 , 47 , 2 ] have been widely adopted. These methods apply geometric transformations, color perturbations, and sample mixing to increase training diversity. However, they operate only on existing samples and cannot generate fundamentally new instances with novel semantics. As a result, they are insufficient for addressing performance degradation caused by data scarcity and semantic imbalance. Compared to GANs [ 10 , 8 , 49 , 20 ] , the diffusion models [ 30 ] achieve superior image quality and training stability for more flexible text-conditioned image synthesis. Layout-to-Image techniques [ 21 , 46 , 41 ] further enhance controllability by conditioning on layout information, such as bounding boxes, and have been explored to synthesize detection-specific training data [ 34 ] . However, applying these methods directly to the maritime domain remains problematic. Maritime scenes demand an explicit semantic understanding of the interaction between objects and their aquatic surroundings, as objects and environmental context (e.g., sea states, reflections, lighting) are closely intertwined in both appearance and meaning. Without modeling this relationship, generative models often produce semantically inconsistent and physically implausible artifacts, such as ships floating unnaturally in mid-air or disconnected from the water surface. Furthermore, existing approaches overlook the fact that synthetic samples vary in training utility due to differences in category, viewpoint, and condition. Failing to account for such disparities leads to suboptimal data selection and limited gains. In this paper, we introduce Neptune-X 1 1 1 The model name “Neptune-X” integrates the marine symbolism of Neptune (Roman sea god) with multi-modality conditional guidance (X). , a unified data generation-selection paradigm designed to address the dual challenges of data scarcity and limited diversity in maritime object detection. Our approach fuses multi-condition maritime scene generation with task-aware sample selection to enhance both quantity and quality of training data. In the generation phase, we develop a controllable generative framework that supports diverse input modalities and produces semantically rich maritime scenes. A central innovation is the Bidirectional Object-Water Attention (BiOW-Attn) mechanism, which explicitly models interactions between objects and their aquatic surroundings to improve the realism and coherence of object placement. This enables the generation of visually plausible maritime scenes with fine-grained spatial semantics (Fig. 1 a). For data selection, Neptune-X incorporates an Attribute-dependent Active Sampling (AAS) strategy to prioritize training samples that are most beneficial for detection performance. This strategy is guided by Attribute-related Training Difficulty Factors (ATDF), which estimate the learning difficulty associated with different semantic attributes, such as viewpoint, object category, and environmental condition. As illustrated in Fig. 1 b, ATDF captures the relative training complexity across attributes and informs the weighting of synthetic samples during selection. By aligning sample value with task difficulty, AAS enables more focused and efficient use of generated data, ultimately guiding the detector to learn from challenging and underrepresented cases. To support training and evaluation under diverse maritime conditions, we construct a new benchmark, the Maritime Generation Dataset, which covers a broad range of scenarios with variations in object category, viewpoint, environment, and location. Extensive experiments demonstrate the effectiveness of Neptune-X in both synthetic scene quality and downstream detection performance, particularly in challenging and underrepresented cases. In summary, our main contributions are threefold: • We present the X-to-Maritime generation framework for maritime scenes, featuring a Bidirectional Object-Water Attention module that enhances realism by modeling object-water interactions under multi-condition inputs. • We propose an Attribute-dependent Active Sampling strategy that estimates training difficulty across semantic dimensions and selects high-value samples through difficulty-aware weighting. • We construct a Maritime Generation Dataset, the first generative benchmark for maritime detection. Experiments show that our method improves both generation quality and detection performance in challenging scenarios. 2 Related Work Diffusion-based Image Generation. Diffusion models [ 14 ] have demonstrated strong cross-modal generation capabilities and have been widely adopted for image synthesis tasks. Early works such as DALL-E 2 [ 26 ] employed hierarchical diffusion guided by CLIP text encoders [ 25 ] to achieve text-to-image generation, while Imagen [ 31 ] further improves generation quality by leveraging the language understanding power of large-scale language models. Stable Diffusion (SD) [ 30 ] made this technology more accessible by introducing latent space compression, enabling efficient high-resolution synthesis. However, text-only conditioning often lacks fine-grained spatial

Read the complete text