Benchmarking Vision-Based Object Tracking for USVs in Complex Maritime Environments
Muhayy Ud Din, Ahsan B. Bakht, Waseem Akram, Yihao Dong, Lakmal Seneviratne, Irfan Hussain · 2024
Read Benchmarking Vision-Based Object Tracking for USVs in Complex Maritime Environments by Muhayy Ud Din, Ahsan B. Bakht, Waseem Akram, Yihao Dong, Lakmal Seneviratne, Irfan Hussain (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07392.txt.
Introduction
Muhayy Ud Din and co-authors benchmark vision trackers and steering rules for robot boats in rough seas, in a paper from two thousand twenty-four. Tracking from a moving boat fights camera shake, dim views, and fast scale change. Seven deep trackers were run on fake and real sea data, paired with mixed steering rules.
A transformer tracker named SeqTrack held up best in harsh states like dust storms. Among steering rules a quadratic regulator gave the smoothest, steadiest hold. Live sea trials backed the lab results. The full frame ties tracker choice to boat control, not sight alone. Readers who build sea robots get tested picks for both halves of the job.
Catalog record
| CREATOR | Muhayy Ud Din, Ahsan B. Bakht, Waseem Akram, Yihao Dong, Lakmal Seneviratne, Irfan Hussain |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-07392
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Benchmarking Vision-Based Object Tracking for USVs in Complex Maritime Environments Muhayy Ud Din, Ahsan B. Bakht, Waseem Akram, Yihao Dong, Lakmal Seneviratne, Irfan Hussain Abstract Vision-based target tracking is crucial for unmanned surface vehicles (USVs) to perform tasks such as inspection, monitoring, and surveillance. However, real-time tracking in complex maritime environments is challenging due to dynamic camera movement, low visibility, and scale variation. Typically, object detection methods combined with filtering techniques are commonly used for tracking, but they often lack robustness, particularly in the presence of camera motion and missed detections. Although advanced tracking methods have been proposed recently, their application in maritime scenarios is limited. To address this gap, this study proposes a vision-guided object-tracking framework for USVs, integrating state-of-the-art tracking algorithms with low-level control systems to enable precise tracking in dynamic maritime environments. We benchmarked the performance of seven distinct trackers, developed using advanced deep learning techniques such as Siamese Networks and Transformers, by evaluating them on both simulated and real-world maritime datasets. In addition, we evaluated the robustness of various control algorithms in conjunction with these tracking systems. The proposed framework was validated through simulations and real-world sea experiments, demonstrating its effectiveness in handling dynamic maritime conditions. The results show that SeqTrack, a Transformer-based tracker, performed best in adverse conditions, such as dust storms. Among the control algorithms evaluated, the linear quadratic regulator controller (LQR) demonstrated the most robust and smooth control, allowing for stable tracking of the USV. Videos and code can be found here: https://muhayyuddin.github.io/tracking/ . I Introduction The use of unmanned surface vessels (USVs) to perform maritime operations has increased significantly. It provides a variety of functionalities for applications like search and rescue operations [ 1 ] [ 2 ] , maritime security [ 3 ] [ 4 ] , and environmental monitoring [ 5 ] [ 6 ] . These vessels are becoming increasingly important in the marine industry due to their ability to operate autonomously, even in challenging conditions such as harsh weather conditions and in areas where human life is at risk. One of the critical features in these environments is autonomous target tracking, which ensures that USVs can effectively monitor and follow objects [ 7 ] . This functionality is essential for tasks such as inspection, data collection, and navigation. To navigate through challenging sea conditions, a robust and reliable control system is required. Various USV control systems have been proposed such as; Proportional-Integral-Derivative (PID), Sliding Mode Control (SMC), and Linear Quadratic Regulator (LQR) for robust navigation and tracking. Target tracking approaches are typically divided into two main categories: filters-based and deep learning-based approaches [ 8 ] . Filter-based tracking techniques, such as Kalman Filter or Particle Filter [ 9 ] , are often applied to track under target in relatively calm sea conditions. Although these techniques are computationally efficient and suitable for real-time applications, they can struggle under highly dynamic marine conditions. Deep learning-based approaches have significantly improved target tracking. These techniques include Convolutional Neural Networks (CNNs) that effectively extract spatial features for object detection [ 10 ] , while Recurrent Neural Networks (RNNs) capture temporal dependencies that enable reliable tracking of moving objects in unpredictable conditions [ 11 ] . Another approach consists of Siamese networks that enable fast and efficient tracking by comparing features across frames [ 12 ] . Furthermore, Transformer-based techniques utilize attention mechanisms to enhance performance by focusing on key features while filtering out distractions such as reflections [ 13 ] . Although these methods show significant performance improvements in tracking, their use in real-time marine applications remains limited. Instead of trackers, object detectors, such as YOLO [ 14 ] and SSD [ 14 ] are mostly used in maritime environments in conjunction with filtering techniques for tracking [ 15 ] . Trackers are mainly designed for static camera setups where the camera is fixed and objects move within the environment. In maritime conditions, however, the camera is mounted on a USV that experiences dynamic motion due to both the vessel’s movement and the ocean waves. These disturbances result in constantly altering the target’s position within the captured image frame. Furthermore, water splashes on the camera often blur the captured image, making detection and tracking significantly difficult. The application of these advanced tracking techniques in such dynamic real-time scenarios has not been extensively explored [ 16 ] . To address this challenge, this study proposes a vision-guided object-tracking framework to explore the performance of deep learning-based trackers in the maritime domain. The framework integrates state-of-the-art object-tracking algorithms with a low-level control algorithm to enable precise target tracking in complex maritime environments. Fig. 1 shows the snapshots of the experiments in a real-world maritime environment. It is important to note that the framework is designed to be generalized, allowing any state-of-the-art tracker to be incorporated as a module. In this study, we benchmark the performance of five distinct trackers, each utilizing different core algorithms such as; CNNs, Siamese networks, or Transformers in real-time marine applications. The key contributions of this work are summarized as follows: Fig. 1: Demonstration of Unmanned Surface Vehicle tracking a target in a real-world maritime experiment. The left images highlight the USV’s navigation toward the target, while the right images show the system’s tracking output. Contribution: The key contribution of this work is a framework for vision-guided real-time object tracking specifically adapted for complex maritime environments. This framework is designed to address the specific challenges posed by dynamic and unpredictable conditions at sea. The contributions of the work are further detailed as follows: • Benchmarking of State-of-the-Art Trackers: We perform a comprehensive evaluation of current state-of-the-art object tracking algorithms, focusing on their performance in maritime environments. This evaluation is based on a custom dataset that includes both simulated and real-world data, ensuring that the benchmarks reflect real maritime challenges such as lighting variations, reflections, and disturbances caused by waves. • Integration with Control Algorithms: The robustness of control algorithms is comprehinsively tested in combination with different tracking systems. By assessing how various trackers influence control strategies, we provide insights into optimizing navigation and object-following tasks for USV under real-world conditions. • Validation in Simulation and Real Sea Conditions: To ensure the robustness and applicability of the proposed framework, we validate the object tracking and control algorithms through in simulation and in real-world sea experiments. These validations demonstrate the system’s ability to operate effectively in dynamic maritime environments. II Related work Traditionally, tracking in maritime environments has relied heavily on radar-based systems [ 17 ] [ 18 ] . While these systems are robust and reliable, however, they are very expensive and have several drawbacks, such as; their high power consumption, which is especially critical for energy-efficient USVs that operate autonomously for extended periods. In addition, radar systems tend to struggle with detecting smaller or low-reflective objects, which are common in maritime settings [ 19 ] [ 20 ] . Recently, vision-based tracking techniques has gained much attention as an alternative tracking approach. The challenges faced in target tracking can be broadly categorized into four types: change in target shape, change in target scale, occlusion, disappearance of the target, and blurred image [ 21 ] . Another key challenge, particularly in dynamic