Real-Time Navigation for Autonomous Aerial Vehicles Using Video

Khizar Anjum, Parul Pandey, Vidyasagar Sadhu, Roberto Tron, Dario Pompili · 2025

Read Real-Time Navigation for Autonomous Aerial Vehicles Using Video by Khizar Anjum, Parul Pandey, Vidyasagar Sadhu, Roberto Tron, Dario Pompili (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01996.txt.

Introduction

Khizar Anjum and co-authors propose a lighter way for drones to navigate by camera, skipping the costly step of building full 3D point clouds. Their 2025 paper instead steers by meaning: road signs and other semantic cues guide the craft. The catch is that spotting such cues takes vision algorithms, which strain a small drone's limited onboard resources.

A Markov decision process framework cuts that workload, deciding when detection is worth running. The team applied it to both feature based and neural network based detectors. Open and closed loop simulations, plus hardware in the loop tests, showed clear gains in energy use and speed. Accuracy dipped only slightly against static and neural baselines.

Catalog record

CREATORKhizar Anjum, Parul Pandey, Vidyasagar Sadhu, Roberto Tron, Dario Pompili
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01996

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Real-Time Navigation for Autonomous Aerial Vehicles Using Video Khizar Anjum, Parul Pandey, Vidyasagar Sadhu, Roberto Tron, Dario Pompili Abstract Most applications in autonomous navigation using mounted cameras rely on the construction and processing of geometric 3D point clouds, which is an expensive process. However, there is another simpler way to make a space navigable quickly: to use semantic information (e.g., traffic signs) to guide the agent. However, detecting and acting on semantic information involves Computer Vision (CV) algorithms such as object detection, which themselves are demanding for agents such as aerial drones with limited onboard resources. To solve this problem, we introduce a novel Markov Decision Process (MDP) framework to reduce the workload of these CV approaches. We apply our proposed framework to both feature-based and neural-network-based object-detection tasks, using open-loop and closed-loop simulations as well as hardware-in-the-loop emulations. These holistic tests show significant benefits in energy consumption and speed with only a limited loss in accuracy compared to models based on static features and neural networks. 1 Introduction Traditional approaches for autonomous navigation are based on accurate geometric knowledge of the environment [ 19 , 6 ] However, these geometric solutions (occupancy grids, point clouds, etc.) could be improved by adding semantic information using techniques such as Computer Vision (CV) techniques. This is especially true for robots working in environments involving close contact with humans (e.g., an indoor environment [ 35 ] or in a search and rescue mission [ 26 ] , etc.), because such semantic information can be interpretable for both humans and robots, allowing for a richer understanding of the environment by the robots, and could also be used by humans to better understand or anticipate the actions taken by a robot. As an example, robots, such as drones, can use the same infrastructure that was developed for humans to navigate. This information can be processed using CV techniques. For example, if a robot detects a “Turn Right" traffic sign, it might mean that the robot should turn right to avoid a corridor in which humans could be present. However, for this approach to navigation to work, it is imperative that the robot can detect semantic information (signs) in near real time . This poses a major challenge, as vision algorithms are generally too complex for robots with limited on-board resources. To counteract such limitations, cloud offloading has been proposed in applications such as localization [ 10 , 22 ] and object recognition [ 14 , 39 ] . However, such approaches struggle in the face of high latency or limited network connectivity, leading to performance deterioration. This is intolerable, as a delayed result might not be useful to the robot as the conditions may have changed. In the worst case, a delayed response may result in injury (i.e., when drones work in the proximity of humans). Motivation: Hence, real-time applications such as autonomous indoor navigation require a better solution than only offloading to a remote server. We propose a lightweight method for executing such algorithms onboard the drone instead of offloading as an alternative. In this way, we get real-time performance and save energy by eliminating complete dependence on the cloud. We do this by running sign-recognition algorithms using local resources if cloud resources are absent or have high latency. Furthermore, since most of the power on drones is already used by rotors to keep them afloat, a small computational load, if used as an alternative to cloud offloading, should not affect mission parameters much. Sign recognition requires detection algorithms from computer vision, which have seen tremendous performance improvement in the last decade; however, this improvement has been made mostly in terms of accuracy and not much in terms of lower-resource usage. By ignoring resource usage, there has been a proliferation of implementation strategies that focus only on achieving the highest accuracy possible. To make such solutions more amenable to on-board processing, we propose to use machine learning methods with different levels of approximations that are selected based on the nature of input data (clutter level, lighting level, etc.), enabling both energy and time savings with respect to a standard one-size-fits-all approach. While image enhancement techniques using generative methods [ 13 , 17 ] have been proposed to address such input quality issues, these solutions remain impractical for resource-constrained devices. Proposed Approach: This work aims to enhance the responsiveness of detection algorithms while maintaining real-time performance. By optimizing execution speed, we can minimize power draw from the drone’s battery, thereby maximizing potential flight duration. Our key insight is that computer vision algorithm selection and parameter tuning should be dynamically adapted based on three key factors: the characteristics of incoming visual data, the available computational and power resources on the device, and the minimum accuracy requirements specified by the application. To achieve this adaptive optimization, we leverage principles from approximate computing . Our methodology carefully balances computational approximations to reduce processing time while maintaining detection accuracy within acceptable bounds. The core of our approach is an intelligent decision system that dynamically selects algorithm parameters to achieve "sufficient" accuracy - delivering acceptable detection performance while significantly reducing computational overhead and energy usage. We implement this decision logic using a Markov Decision Process (MDP) framework to optimize parameter selection across video frame sequences. The MDP approach was chosen for its lightweight computational footprint and its ability to provide transparent, interpretable decision-making behavior. Contributions: Our work advances lightweight machine learning and decision-making in several ways: 1. We demonstrate the versatility of our methodology across different machine learning paradigms, including traditional feature-based approaches (Histogram of Gradients and Edgeboxes with SVMs) and modern deep learning methods (CNNs). 2. We validate our framework through comprehensive testing, progressing from static image analysis to dynamic video processing and ultimately to closed-loop control systems. Our research yielded the following key results : 1. Through systematic offline analysis, we identified optimal algorithm parameters that achieve a 42\% speedup while preserving 80\% detection accuracy. 2. Testing on standard benchmarks and custom drone-captured footage demonstrated real-time performance at 30\,\mathrm{fps} for 480\times 270 resolution, with accuracy degradation under 2\% compared to non-optimized parameters. 3. We verified practical applicability through hardware-in-the-loop simulations, where a resource-constrained drone successfully navigated custom environments using our framework. Generalizability: Though we focus on object detection as our primary use case, our MDP-based parameter optimization approach represents a general framework applicable to diverse algorithmic domains requiring real-time performance under resource constraints. Article Organization: The remainder of this paper is structured as follows: Sect. 2 contextualizes our work within existing literature. Sect. 3 presents our decision framework, detailing the MDP formulation and its application to object detection. Sect. 4 describes our experimental methodology and results. Finally, Sect. 5 summarizes our findings and discusses future directions. 2 Related Work This section examines existing research in mobile and approximate computing for resource-constrained environments, as well as applications of MDPs in computer vision tasks. Mobile Computation Offloading: Contemporary mobile applications leveraging object detection, including interactive games, landmark identification systems [ 8 ] , and facial analysis tools [ 5 ] , frequently delegate computationally intensive operations to remote cloud infrastructure. While this approach enables sophisticated functionality on modest hardware, it introduces significant drawbacks for real-time applications - notably increased latency,

Read the complete text