Strong Baseline: Multi-UAV Tracking via YOLOv12 with BoT-SORT-ReID

Yu-Hsi Chen · 2025

Read Strong Baseline: Multi-UAV Tracking via YOLOv12 with BoT-SORT-ReID by Yu-Hsi Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17237.txt.

Introduction

Yu-Hsi Chen assembles a strong, simple tracker for multiple drones in thermal video. Heat cameras see drones as faint, tiny specks against noisy backgrounds. Contrast is low and targets are small, so both spotting and following are hard. Chen pairs a recent detector, YOLOv12, with the BoT-SORT tracker and re-identification.

Tailored training and inference choices do the quiet work. Judged by the fourth Anti-UAV Challenge measures, the system reached competitive performance. Notably, it used no contrast enhancement and no fusion across time to fluff the targets. That restraint is the point: a clean baseline others can build on. The paper details the implementation, analyzes the experiments, and discusses next improvements. Code is released for anyone to reproduce or extend.

Catalog record

CREATORYu-Hsi Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17237

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Strong Baseline: Multi-UAV Tracking via YOLOv12 with BoT-SORT-ReID Yu-Hsi Chen Abstract Detecting and tracking multiple unmanned aerial vehicles (UAVs) in thermal infrared video is inherently challenging due to low contrast, environmental noise, and small target sizes. This paper provides a straightforward approach to address multi-UAV tracking in thermal infrared video, leveraging recent advances in detection and tracking. Instead of relying on the well-established YOLOv5 with DeepSORT combination, we present a tracking framework built on YOLOv12 and BoT-SORT, enhanced with tailored training and inference strategies. We evaluate our approach following the 4th Anti-UAV Challenge metrics and reach competitive performance. Notably, we achieved strong results without using contrast enhancement or temporal information fusion to enrich UAV features, highlighting our approach as a ”Strong Baseline” for multi-UAV tracking tasks. We provide implementation details, in-depth experimental analysis, and a discussion of potential improvements. The code is available at https://github.com/wish44165/YOLOv12-BoT-SORT-ReID . 1 Introduction Multi-UAV tracking has emerged as a crucial application in recent years, driven by significant advancements in hardware, detection models, and tracking algorithms. As UAVs equipped with sophisticated visual systems and advanced control dynamics continue to proliferate, a wide range of UAV-based products have been introduced, as presented in Nyboe et al. 2022 . However, these innovations also introduce new challenges, particularly in tracking UAV swarms. The need for effective swarm tracking has become increasingly urgent due to growing security concerns and the rising threat posed by unauthorized UAVs. Various UAV-related datasets have been developed to address these challenges to advance tracking and detection tasks. These datasets include trajectory reconstruction datasets, such as those in Li et al. 2020 ; Hwang et al. 2023 , which provide UAV trajectories captured from single or multi-view cameras, and trajectory-based UAV datasets introduced in Chen 2024 . Additionally, RGB-based footage datasets, including those in Pawełczyk and Wojtyra 2020 ; Steininger et al. 2021 ; Li et al. 2016 ; Registry 2023 , have been widely used. Among these, thermal infrared video-based UAV datasets featuring both single-object tracking (SOT) and multi-object tracking (MOT) scenarios in Jiang et al. 2021 ; Huang et al. 2023 , have gained significant attention, particularly in major challenge events. These datasets have played a pivotal role in improving UAV tracking and detection capabilities. Thermal infrared videos offer advantages over traditional RGB imagery, such as enhanced visibility in low-light and adverse weather conditions, making them ideal for security and surveillance applications. This paper focuses on using thermal infrared video for multi-UAV tracking, exploiting its importance in challenging environments where RGB-based methods may fail. Fig. 1 (a) illustrates thermal infrared frames with diverse backgrounds from the MOT training set, while Fig. 1 (b) highlights minor defects, such as annotation errors, redundancies, missed labels, and low-quality frames, that account for a negligible portion of the dataset and can be safely disregarded during training. Additionally, Fig. 2 displays cropped image patches from bounding box annotations in the training set, illustrating the varying sizes of UAVs, from several pixels to single-digit pixels. We build a complete UAV tracking workflow by leveraging the latest YOLOv12 Tian et al. 2025 detector and BoT-SORT Aharon et al. 2022 tracking algorithm, which outperform the YOLOv5 Jocher et al. 2020 with the DeepSORT Wojke et al. 2017 pipeline. We also implement some strategies to enhance multi-UAV tracking performance further. Our contributions are as follows: 1. We establish a multi-UAV tracking workflow based on YOLOv12 and BoT-SORT, setting a strong baseline for thermal infrared video-based multi-UAV tracking tasks. 2. We provide insightful analysis of various trial adjustments, such as the impact of input image size and tracker buffer tuning, and offer essential considerations for future improvements starting from our strong baseline. Figure 1 : Demonstration using training and testing data from Track 3. (a) Shows UAV swarms with varying sizes and backgrounds in the training data. (b) Highlights annotation errors and frame defects: MultiUAV-230 (train) has incorrect annotations, MultiUAV-256 (train) contains redundant annotations, MultiUAV-294 (train) has missed annotations, and MultiUAV-068 (test) includes a poor-quality frame. Figure 2 : Illustration of cropped image patches from training data annotations. Each patch corresponds to a bounding box from MultiUAV-002, MultiUAV-013, MultiUAV-087, and MultiUAV-223, with sizes approximately 28\times 24 , 10\times 10 , 6\times 6 , and 11\times 10 , respectively. The top number denotes the frame sequence (1st to 12th frames), and each row represents the same object (same ID across frames). 2 Related Work Existing perspectives for improving thermal infrared video-based multi-UAV tracking can be categorized into annotation and benchmarking, spatial information enhancement, temporal and motion modeling, real-time optimization, unified frameworks, and detection-based tracking systems. As high-quality annotation is fundamental for robust tracking, prior studies have examined the impact of annotation errors on object detection Koksal et al. 2020 , incorporated Multiple Hypothesis Tracking (MHT) to leverage temporal cues and reduce false positives Ince et al. 2021 , and introduced benchmarks to evaluate detection and tracking methods on UAV datasets Isaac-Medina et al. 2021 . Spatial information enhancement techniques, such as the Image Pyramid Guidance (IPG) module presented in Liu et al. 2020 , address feature imbalance by preserving fine-grained spatial details for accurate bounding box regression and classification, even in deep network layers. To further improve tracking robustness, temporal and motion modeling techniques exploit inter-frame correlations, enhancing continuity and reducing fragmentation Huang et al. 2021 ; He et al. 2023 ; Li et al. 2023 ; Yang et al. 2023 . Complementary to this, real-time optimization strategies reduce inference latency while maintaining accuracy, enabling efficient UAV tracking in real-world applications Wang et al. 2020 ; Wu et al. 2020 ; Fang et al. 2021 ; Lyu et al. 2023 . Beyond these, unified frameworks integrate detection and tracking into end-to-end solutions, streamlining multi-UAV tracking pipelines Zhao et al. 2021 ; Yu et al. 2023 . Additionally, detection-based methods incorporating cascading post-processing modules refine tracking accuracy by mitigating false positives and improving localization Tang et al. 2023 . While prior works have contributed significantly to multi-UAV tracking, our approach advances the field by leveraging the latest detector and tracker, setting a new benchmark for thermal infrared video-based UAV tracking and guiding future research in the multi-UAV tracking task. 3 Methodology This section first defines the problem scope, followed by data analysis and preparation for model training. We then introduce the primary detection model, YOLOv12, and the tracking algorithm, BoT-SORT, before detailing our training and inference strategies. 3.1 Problem Statement The goal is to track UAVs as accurately as possible, with evaluation metrics detailed in Section 4.1 . The challenge consists of three tracks, each corresponding to a different scenario. Tracks 1 and 2 are SOT tasks, differing in whether the UAV’s initial location is given. Track 3 is a MOT task where the initial locations of UAVs are provided. 3.2 Data Analysis and Preparation We first analyze each track’s training and testing data, as summarized in Table 1 . Tracks 1 and 2 share the same training set, consisting of 23 sequences at 512 \times 512 resolution and 200 sequences at 640 \times 512 resolution. The training set for Track 3 is composed of 200 sequences at 640 \times 512 resolution. For testing, Tracks 1 and 2 each contain 216 non-overlapping sequences. Track 1’s test

Read the complete text