Advising Agent for Supporting Human-Multi-Drone Team Collaboration

Hodaya Barr, Dror Levy, Ariel Rosenfeld, Oleg Maksimov, Sarit Kraus · 2025

Read Advising Agent for Supporting Human-Multi-Drone Team Collaboration by Hodaya Barr, Dror Levy, Ariel Rosenfeld, Oleg Maksimov, Sarit Kraus (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17960.txt.

Introduction

Hodaya Barr, Dror Levy, Ariel Rosenfeld, Oleg Maksimov, and Sarit Kraus design a software adviser for people who run teams of drones, in a paper published in two thousand twenty-five. Fleets of drones now serve many industries, but their autonomy is still limited, so human operators must supervise them.

That supervision is demanding. An operator must watch drone states and sensor feeds, decide in real time, and adapt as conditions shift. Mistakes or slow calls can drag down the whole team's work. In search and rescue missions, the cost of a poor call can be severe. The authors add an advising agent that supports the human during the mission. The agent is meant to improve joint decisions without taking control away from the operator. The paper sets out the design and its role in human-multi-drone teamwork.

Catalog record

CREATORHodaya Barr, Dror Levy, Ariel Rosenfeld, Oleg Maksimov, Sarit Kraus
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-17960

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Advising Agent for Supporting Human-Multi-Drone Team Collaboration Hodaya Barr, Dror Levy, Ariel Rosenfeld, Oleg Maksimov, Sarit Kraus Abstract Multi-drone systems have become transformative technologies across various industries, offering innovative applications. However, despite significant advancements, their autonomous capabilities remain inherently limited. As a result, human operators are often essential for supervising and controlling these systems, creating what is referred to as a human-multi-drone team. In realistic settings, human operators must make real-time decisions while addressing a variety of signals, such as drone statuses and sensor readings, and adapting to dynamic conditions and uncertainty. This complexity may lead to suboptimal operations, potentially compromising the overall effectiveness of the team. In critical contexts like Search And Rescue (SAR) missions, such inefficiencies can have costly consequences. This work introduces an advising agent designed to enhance collaboration in human-multi-drone teams, with a specific focus on SAR scenarios. The advising agent is designed to assist the human operator by suggesting contextual actions worth taking. To that end, the agent employs a novel computation technique that relies on a small set of human demonstrations to generate varying realistic human-like trajectories. These trajectories are then generalized using machine learning for fast and accurate predictions of the long-term effects of different advice. Through human evaluations, we demonstrate that our approach delivers high-quality assistance, resulting in significantly improved performance compared to baseline conditions. I Introduction Multi-drone systems are revolutionizing various industries including agriculture [ 46 ] , construction [ 40 ] , and search and rescue (SAR) [ 29 ] , to name a few [ 19 ] . As drone technology advances, the potential for innovative applications and their societal impact is expected to continue to grow [ 44 ] . Nevertheless, there are some inherent limitations to what multi-drone systems can handle autonomously. As such, it is common for human operators to provide an extra layer of monitoring, decision-making, and situational awareness, thus forming a so-called “human-multi-drone team” [ 38 , 15 ] . Supervising and controlling multiple drones in dynamic and uncertain environments is a highly challenging task [ 34 ] . Model-driven approaches have been proposed to facilitate human-on-the-loop drone missions, where autonomous decisions are made by drones but guided by human oversight to improve mission success in time-critical operations [ 2 ] . In particular, the human operator is typically expected to monitor and respond to the drones’ locations, statuses, sensor readings, and various other signals and information in a timely fashion while managing unexpected events, emergencies, and changing circumstances. Research by [ 35 ] demonstrated that when control tasks are extensive, human operators can effectively collaborate with two drones. However, even experienced operators encountered difficulties managing three drones (or more) in complex settings, leading to a reduction in the overall team performance. The integration of intelligent agents into mixed teams is becoming increasingly prevalent, offering new opportunities and challenges in team dynamics [ 23 , 32 ] . Aligned with this important line of work, in this paper, we develop and evaluate a novel advising agent for supporting human-multi-drone team collaboration for lifelike tasks. In our setting, advice refers to action recommendations aimed at improving the overall team performance, which the advising agent generates based on contextual information. The presented agent relies on machine learning estimations of the long-term effects of various human-like actions which, in turn, are translated into useful advice. Central to our approach is the generation of realistic trajectories that mimic human operators, using very few human demonstrations. Specifically, in order to adequately utilize machine learning to estimate the long-term effects of different actions, we propose a method to simulate the trajectories resulting from (unseen) human-like actions from a given setting using a limited number of human-generated examples. We instantiate our approach to a realistic SAR simulation, and through an extensive human study, we show that our agent is capable of providing high-quality advice, which, in turn, brings about favorable team performance compared to baseline conditions. II Related Work Various techniques have been proposed to assist human operators in managing multiple drones. Roughly speaking, these can be divided into two groups: Intelligent User Interfaces (IUIs) and intelligent advising agents. The former group typically seeks to assist the operator in gaining a better situational awareness and understanding and providing more efficient command and control capabilities. Several prior works explore natural user interfaces, such as speech, gestures, body position, and eye-tracking, to facilitate intuitive drone interaction (e.g., [ 14 , 50 , 48 ] ). [ 1 ] examines how different IUI design choices affect human situational awareness, while [ 10 ] proposes a task-based graphical interface to enhance control and situational awareness. Similarly, [ 25 ] explores classifying operator functional states using physiological data to dynamically improve human-machine interaction in drone swarm piloting. Boggess et al. [ 7 , 6 ] propose XAI methods to explain the behavior and decision-making processes of a team of robots to a human collaborator. Previous research has introduced methods to improve interaction between a single automated team member (e.g., a robot) and a human operator to enhance overall collaboration [ 13 , 41 , 5 , 4 ] . Projects like HADRON have developed AI-driven control systems to reduce human workload by improving tasks such as target and defect detection [ 9 ] . [ 3 ] proposed a forward simulation-based alert system to notify supervisors of potential negative events. In [ 49 ] , natural language processed by LLMs allows human operators to communicate with a drone team, though no human studies were conducted, and timely comment generation was not evaluated. We focus on intelligent advising agents that seek to support the operator by advising the operator on which actions to take [ 39 ] . This approach complements the previously discussed methods, as it is interface-agnostic (i.e., capable of being integrated with any user interface) and independent of the AI tools used by the drones. It is shown to be highly effective (i.e., better team performance). The intelligent advising agents on which actions to take approach was explored in several domains; for example, [ 45 ] provides an intelligent agent for the teleoperation of autonomous vehicles, and [ 47 ] developed an intelligent agent for a multi-vehicle setting in order to study ways to increase user trust. Most closely related to our work is [ 39 ] , who tackled a similar challenge in a non-aerial environment. The authors suggested the use of a so-called “utopic environment” where they simulate the effects of different actions by assuming their ground robots operate optimally, never malfunction, and require no supervision or control from the human operator. In other words, the authors assume that the utopically-simulated long-term effects of an action are a reasonable approximation of the actual long-term effects. The rationale for using such a utopic environment is clear - it makes the collection and annotation of extensive human demonstrations unnecessary, as these simulations execute fully autonomously. Unfortunately, this assumption is highly unrealistic in complex real-world environments, as noted by the original authors themselves. In particular, when considering multiple drones deployed in unexpected dynamic conditions that require human intervention, this oversimplifying assumption may lead to poor estimations and sub-optimal advice provision. In this work, we propose

Read the complete text