FIMD: Fast Isolated Marker Detection for UV-Based Visual Relative Localisation in Agile UAV Swarms

Vojtěch Vrba, Viktor Walter, Petr Štěpán, Martin Saska · 2025

Read FIMD: Fast Isolated Marker Detection for UV-Based Visual Relative Localisation in Agile UAV Swarms by Vojtěch Vrba, Viktor Walter, Petr Štěpán, Martin Saska (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19663.txt.

Introduction

Vojtěch Vrba and three co-authors speed up the marker detection that lets drone swarms locate each other visually. In agile unmanned aerial vehicle swarms, each craft finds its teammates by spotting ultraviolet markers in camera frames, and that detection has to run onboard in real time. The team re-engineered the detector three ways: an optimized CPU routine, a GPU shader, and an FPGA streaming design.

The CPU and GPU versions cut the average processing time per pixel by two to three orders of magnitude compared with the unoptimized state of the art. The FPGA version minimized the total delay from camera exposure to detection result. Tests on several embedded platforms showed the approach works even on low-end drones.

Catalog record

CREATORVojtěch Vrba, Viktor Walter, Petr Štěpán, Martin Saska
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19663

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FIMD: Fast Isolated Marker Detection for UV-Based Visual Relative Localisation in Agile UAV Swarms Vojtěch Vrba, Viktor Walter, Petr Štěpán, Martin Saska Abstract A novel approach for the fast onboard detection of isolated markers for visual relative localisation of multiple teammates in agile UAV swarms is introduced in this paper. As the detection forms a key component of real-time localisation systems, a three-fold innovation is presented, consisting of an optimised procedure for CPUs, a GPU shader program, and a functionally equivalent FPGA streaming architecture. For the proposed CPU and GPU solutions, the mean processing time per pixel of input camera frames was accelerated by two to three orders of magnitude compared to the unoptimised state-of-the-art approach. For the localisation task, the proposed FPGA architecture offered the most significant overall acceleration by minimising the total delay from camera exposure to detection results. Additionally, the proposed solutions were evaluated on various 32-bit and 64-bit embedded platforms to demonstrate their efficiency, as well as their feasibility for applications using low-end UAVs and MAVs. Thus, it has become a crucial enabling technology for agile UAV swarming. keywords Marker Detection, Visual Relative Localisation, Agile UAV Swarming, FPGA, Embedded Systems 1 Introduction A robust and accurate localisation system is a key component for the close interaction and collaboration of Unmanned Aerial Vehicles (UAVs). Such a system is an essential tool for the fulfilment of various related objectives, such as mutual collision avoidance, swarm cohesion and alignment, and multi-agent trajectory planning ( Yasin et al. 2020 ) . Figure 1 : Real-world application of the UVDAR system. The commonly used localisation systems are dependent on external infrastructure, including motion capture (MoCap) systems ( Abdelkader et al. 2021 ) used for well defined indoor environments, and global navigation satellite systems (GNSS) used for outdoor, open-sky conditions with a direct satellite link ( Lopez et al. 2021 ) . In general, these systems provide an absolute estimate of the poses and velocities of the swarm agents given the corresponding frame of reference. Despite the high precision these methods may provide under ideal conditions, real-world applications usually have to deal with GNSS-denied, obstacle-dense, or unknown environments. Such conditions lead to the need for localisation methods based on relative inter-agent measurements. It was shown that distance-based methods utilising omnidirectional radio-based communication standards, such as Wi-Fi ( Kealy et al. 2019 ) , Bluetooth ( Coppola et al. 2018 ) , or ultra-wideband radio (UWB) ( Goel 2017 ) , are able to provide accurate estimates of relative agent poses. When compared to the former two standards, the UWB-based approach offers the most accurate distance measurements between network nodes. Consequently, recent literature has proposed its fusion with other sensors, such as stereo cameras ( Xu et al. 2022 ) or LIDARs ( Queralta et al. 2022 ) . However, with an increasing number of swarm agents, the probability of depleting the network resources increases. This can possibly result in signal jamming and an overall decrease of the localisation system robustness when approaching wireless network limits. Another category of relative localisation techniques relevant to this work is based on computer vision. This modality offers better scalability and saturation immunity while introducing new challenges. In general, vision-based localisation methods may be either marker-less or they may use additional markers attached to the targets. The former methods perform recognition of individual targets directly in camera frames, usually by employing convolutional neural networks (CNNs) to handle detection of miscellaneous UAV projections ( Vrba et al. 2019 ; Schilling et al. 2021 ) . However, the CNNs must be trained using well-prepared datasets before the actual real-world deployment of the UAVs. The latter methods are based on detection of predefined markers which may be either passive or active. Methods utilising passive markers, such as ( Krajník et al. 2014 ) , suffer primarily from dependency on strong lighting conditions, which prevents their general deployment in diverse environments. Methods based on active markers eliminate this dependency by utilising light-emitting diodes (LEDs) working either in the visible spectrum ( Dias et al. 2016 ; Stuckey et al. 2023 ) or in the infra-red (IR) spectrum ( Cutler et al. 2013 ; Faessler et al. 2014 ; Xun et al. 2023 ) . However, these methods are usually applicable in indoor environments only due to an ubiquitous amount of sunlight outdoors. For outdoor environments, it was shown in ( Walter et al. 2018 ) that the usage of ultra-violet (UV) light is able to overcome this challenge. Figure 2 : The UVDAR system pipeline. (a) UV power LEDs (b) UV-sensitive camera Figure 3 : Components of the UVDAR system. The Ultra-Violet Direction And Ranging (UVDAR) system ( Walter et al. 2018 ; Walter et al. 2018a ; Walter et al. 2019 ) is a vision-based distributed relative localisation framework. The system, illustrated in Fig. 1 , uses UV LEDs as UAV markers together with UV bandpass filters mounted on fisheye camera lenses. Additionally, the UV LEDs of different UAVs emit predefined unique blinking sequences that are used for their identification, as well as for the estimation of their relative poses and flight trajectories. The system consists of a two-part pipeline, as shown in Fig. 3 . The transmitter emits the blinking sequences via an optical channel using UV LEDs (Fig. 3 a). The receiver consists of four essential elements: a monochromatic camera with UV bandpass filters and a fisheye lens (Fig. 3 b), a detector of isolated points in camera frames, a tracker of the blinking sequences between subsequent frames, and a pose estimator used for computation of the locations and orientations of the observed UAVs relative to the observing UAV. The UVDAR system forms an essential localisation tool used by teams of closely collaborating UAV agents. All the necessary computer vision calculations are performed on the onboard computers of the individual UAV agents, while consuming a significant amount of their computational resources. Moreover, with the increasing agility of the UAVs, these requirements tend to grow rapidly while reflecting the requisite increase of the sensors’ frame rate. In this paper, the theoretical background of an unoptimised detection algorithm for the isolated LED markers of the UVDAR system is investigated. Next, its optimised CPU-based variant is presented as a pseudocode, which is easily interpreted as a small set of assembly instructions. Its parallel variants are also presented, i.e., a functionally equivalent GPU shader program and an FPGA streaming architecture. Lastly, the performance of the presented solutions is evaluated along with a state-of-the-art approach on an onboard computer, as well as several other CPU+GPU boards. In the case of the FPGA solution, the influence of the camera interface on the overall performance is also discussed. 1.1 Problem statement This paper tackles the problem of fast real-time visual localisation for agile multi-robot systems, based solely on onboard sensors without any radio communication support. In particular, the main focus is on the insufficient performance of a state-of-the-art computer vision algorithm for the detection of isolated LED markers when high frame rate camera sensors are utilised under the limitations of the onboard computational resources of UAVs. 1.2 Contribution A novel three-fold approach to the fast detection of isolated UV LED markers in camera frames for

Read the complete text