Towards Low-Latency Event-based Obstacle Avoidance on an FPGA-Drone

Pietro Bonazzi, Christian Vogt, Michael Jost, Lyes Khacef, Federico Paredes-Vallés, Michele Magno · 2025

Read Towards Low-Latency Event-based Obstacle Avoidance on an FPGA-Drone by Pietro Bonazzi, Christian Vogt, Michael Jost, Lyes Khacef, Federico Paredes-Vallés, Michele Magno (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10400.txt.

Introduction

Pietro Bonazzi and five co-authors test a fast drone that dodges blocks with a special kind of camera, in a paper from 2025. Event cameras fire only when light changes, which can give much quicker sight than normal frames.

On a chip built for speed, the event model ran at a very high rate and made smaller time and space errors than a normal image model. It also picked dodge moves more soundly, most of all when telling still scenes from moving ones. End to end delay was about two thousandths of a second, which suits tight, fast flight.

Catalog record

CREATORPietro Bonazzi, Christian Vogt, Michael Jost, Lyes Khacef, Federico Paredes-Vallés, Michele Magno
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-10400

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Low-Latency Event-based Obstacle Avoidance on an FPGA-Drone Pietro Bonazzi, Christian Vogt, Michael Jost, Lyes Khacef, Federico Paredes-Vallés, Michele Magno Abstract This work quantitatively evaluates the performance of event-based vision systems (EVS) against conventional RGB-based models for action prediction in collision avoidance on an FPGA accelerator. Our experiments demonstrate that the EVS model achieves a significantly higher effective frame rate (1 kHz) and lower temporal (- 20 ms 20\text{\,}\mathrm{ms} ) and spatial prediction errors (- 20 mm 20\text{\,}\mathrm{mm} ) compared to the RGB-based model, particularly when tested on out-of-distribution data. The EVS model also exhibits superior robustness in selecting optimal evasion maneuvers. In particular, in distinguishing between movement and stationary states, it achieves a 59 percentage point advantage in precision (78% vs. 19%) and a substantially higher F1 score (0.73 vs. 0.06), highlighting the susceptibility of the RGB model to overfitting. Further analysis in different combinations of spatial classes confirms the consistent performance of the EVS model in both test data sets. Finally, we evaluated the system end-to-end and achieved a latency of approximately 2.14 ms 2.14\text{\,}\mathrm{ms} , with event aggregation ( 1 ms 1\text{\,}\mathrm{ms} ) and inference on the processing unit ( 0.94 ms 0.94\text{\,}\mathrm{ms} ) accounting for the largest components. These results underscore the advantages of event-based vision for real-time collision avoidance and demonstrate its potential for deployment in resource-constrained environments. Keywords Dataset, Obstacle Avoidance, Event-Based Cameras. Funding This research was funded by the Swiss National Foundation (219943) and the Sony Research Award Program. Reproducibility Figure 1 : Overview of the proposed DPU-Based system for Obstacle Avoidance . Our method achieves an end-to-end perception latency of sub 3ms. The data set and the code base can be installed following the instructions from the following link: https://github.com/pbonazzi/eva . SoC System on Chip FPGA Field Programmable Gate Array EVS Event Vision Sensor DPU Deep Learning Unit DDR Double Data Rate 1 Introduction Unmanned aerial vehicles (UAVs) are increasingly deployed across diverse applications, including logistics, surveillance, agriculture, and disaster response. As these autonomous systems operate in complex and dynamic environments [ 22 ] , ensuring real-time, reliable obstacle avoidance remains a critical challenge. Existing vision-based approaches predominantly rely on conventional RGB cameras due to their high spatial resolution and compatibility with established computer vision algorithms [ 24 ] . However, these frame-based sensors suffer from inherent limitations such as motion blur, latency, and reduced dynamic range—particularly in high-speed or low-light conditions [ 25 ] . Event-based vision sensors (EVS) offer an alternative paradigm by asynchronously capturing pixel-level intensity changes, enabling ultra-low latency, high temporal resolution, and enhanced robustness to lighting variations [ 8 , 10 , 3 , 7 , 14 ] . This sensing modality has demonstrated promising results in UAV applications, particularly for rapid motion detection and reactive navigation [ 19 , 5 , 2 , 16 ] . Recent advances in event-based vision have led to real-world UAV demonstrators [ 9 ] , but key challenges remain—namely, the need for robust, real-time decision-making in unpredictable environments and efficient onboard processing. Despite the advantages of event-based vision, prior works have primarily focused on obstacle detection rather than direct action prediction for collision avoidance. Moreover, most implementations rely on software-based processing pipelines, which introduce latency and limit real-time responsiveness. Existing FPGA-based UAV architectures have demonstrated efficiency in general AI processing tasks [ 20 , 11 , 13 ] , but few studies have explored FPGA acceleration for event-based action prediction in UAV navigation. The absence of comprehensive benchmarking between event-based and RGB-based methods on FPGA accelerators further limits our understanding of their comparative performance in real-world collision avoidance scenarios. To bridge this gap, we propose an FPGA-event-based processing pipeline designed for real-time action prediction in UAV collision avoidance. Unlike prior works that primarily focus on obstacle detection or trajectory estimation, our approach evaluates event-based and RGB modalities to predict both the position and timing of potential collisions with high precision. We quantitatively evaluated event-based vision systems against conventional RGB-based models for collision avoidance, demonstrating superior temporal resolution and spatial accuracy. Additionally, we provide a comprehensive analysis of event-based action prediction across two test scenarios, including out-of-distribution trajectories, and release all 238 recordings as an open dataset to foster further research. To ensure practical feasibility, we implement an FPGA-based processing pipeline that achieves a low-latency inference time of approximately 2.14\text{\,}\mathrm{ms} , enabling real-time UAV navigation which features an efficient frame aggregation that processes raw EVS ( EVS ) events, accumulating them into 80\times 80 pixel frames with fully configurable accumulation parameters. A visual overview of the proposed system is shown in Fig. 1 . 2 Related Work Recent advancements in UAV technology have increasingly focused on integrating FPGA-based architectures to enhance real-time processing capabilities. This section reviews notable contributions in this domain, highlighting existing research and identifying key differences from our approach. 2.1 Event-Based Obstacle Avoidance Prior research has explored event-based vision systems for UAV obstacle avoidance using both traditional and learning-based approaches. Falanga et al. [ 5 ] and Rodriguez-Gomez et al. [ 18 ] leveraged optical flow and clustering techniques for dynamic obstacle avoidance in quadrotors and ornithopter robots, respectively. Falanga [ 4 ] presented a theoretical framework that jointly considers perception latency and actuation limitations to analyze the maximum speed at which a robot can safely navigate through unknown, cluttered environments, highlighting the trade-offs between sensing latency, sensing range, and actuation capabilities for high-speed navigation. More recent works have shifted towards data-driven solutions: Sanket et al. [ 19 ] and Bhattacharya et al. [ 1 ] developed monocular event-camera-based static obstacle avoidance methods, while Bisulco et al. [ 2 ] introduced spatio-temporal neural networks for fast-motion perception. Additionally, Paredes-Valles et al. [ 16 ] trained a spiking neural network to enable autonomous flight using raw event-based camera data. While these works focus on obstacle detection and navigation, our study extends the scope by evaluating action prediction in a collision avoidance scenario. Unlike prior event-based models, i.e. [ 4 ] , our approach emphasizes real-time decision-making robustness on an FPGA accelerator. Furthermore, we introduce a lightweight inference pipeline and conduct extensive comparisons with RGB-based models, highlighting the advantages of event-based processing in terms of accuracy and robustness. 2.2 FPGA-Based Architectures for UAVs Several studies have explored FPGA-based UAV architectures to improve computational efficiency and response time. Guo et al. [ 11 ] proposed a hybrid UAV architecture integrating FPGA, ARM, and GPU components to overcome hardware constraints and enhance real-time image processing. Similarly, Kovari and Ebeid [ 13 ] introduced MPDrone, an FPGA-based system designed for intelligent real-time drone operations, offloading AI computations to the FPGA while maintaining control processes on a CPU. Tan et al. [ 20 ] focused on FPGA-based acceleration for simultaneous localization and mapping (SLAM), demonstrating efficiency improvements in SLAM processing. In contrast, our work targets action prediction for collision avoidance rather than general UAV computation. We demonstrate that event-based models can outperform RGB-based methods on FPGA accelerators in both speed and accuracy, achieving real-time inference with minimal latency ( 2.14\text{\,}\mathrm{ms} vs estimated 21.14\text{\,}\mathrm{ms} ). Our approach prioritizes not only computational efficiency but also decision-making reliability in out-of-distribution scenarios. 2.3 Open-Source Frameworks and Platforms Nyboe et al. [ 15 ] presented MPSoC4Drones, an open framework

Read the complete text